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ABSTRACT 

This study examines the potential impacts of the DoD emphasizing the use of 
parametric cost estimating methods or techniques in the acquisition process. As one 
of many initiatives to improve the DoD acquisition process through use of 
commercial practices, parametric cost estimating has the potential to be helpful in 
many applications for which it has never before been considered. This study, 
conducted through a questionnaire, personal interviews and a review of recently 
released publications from the DoD Joint Govemment/Industry Initiative, identifies 
areas of interest for those anticipating using parametric cost estimating methods and 
techmques. These areas include the Program Definition & Risk Reduction and 
Engmeering & Manufacturing Development phases of program management as well 
as for many pre-award contract actions. The data from this thesis show that the 
majority of the personnel in the DoD acquisition community believe that parametric 
cost estimating methods can be used effectively in those areas. The data also show 
that the methods also may have applications in the Production, Fielding/Deployment 
& Operational Support phase of program management and post-award contract 
actions such as negotiating changes, forward pricing rate agreements, or analyzing 
claims. 
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INTRODUCTION 



I . 

A. BACKGROUND 

This thesis analyzes Department of Defense (DoD) Business 
Process Reengineering initiatives that focus on increasing the 
use of parametric cost estimating techniques in DoD 
applications. Using parametric pricing techniques to estimate 
program costs has a relatively short history in comparison 
with other pricing techniques. To date, cost estimates based 
on parametric estimating methods have been restricted mainly 
to overall initial planning but they have wider applications 
that are now being contemplated through direct policy action. 
It has been proposed that they be applied to price estimates 
for individual contract actions or families of contract 
actions to reduce a potential contractor's bid and proposal 
costs, proposal cycle time, and to expedite the Federal 
Government proposal evaluation and contract award process. 

B. SCOPE AND ASSUMPTIONS 

The objective of this thesis is to study the policy 
effects of increasing the use of parametric cost estimation 
methods in individual contract actions. Any reference to 
program management uses of parametric cost estimating methods 
is done to relate how individual contract actions fit within 
the framework of a generic program. Anticipated uses in 
specific programs or contract actions are not reviewed or 
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analyzed. No attempt is made to validate any existing 
parametric cost estimating models, techniques, processes, 
software, or their applications in specific situations. 

This thesis identifies potential impediments to the 
implementation of the policy and makes recommendations on how 
they can be overcome. Specifically, the affects on proposal 
generation, evaluation, and subsequent contract performance is 
evaluated. The basic effort is directed at producing an 
accurate picture of how parametric cost estimating techniques 
may be used by the Department of Defense (DoD) and contractors 
throughout the contracting process until completion. Many 
studies have been done on the use of parametric cost estimate 
development for entire classes of equipments, individual 
contract actions, or programs, but none have focused on the 
affects of the policy behind the use of the methods. 

It was assumed at the onset of the research that few 

organizations and personnel employed by DoD either directly or 
/ 

through a contract vehicle were trained in the use of 
parametric cost estimating methods. It was further assumed 
that of those organizations and people trained in their use, 
most would be concentrated at the program office level or 
higher in buying organizations or in the DoD policy making 
areas. Finally, it was assumed that neither the Federal 
Government as a whole nor DoD as a subset had promulgated 
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specific guidance, policy, or instructions regarding the use 
and application of parametric cost estimating techniques. 

C . RESEARCH OBJECTIVE 

This effort is intended to review the increased attention 
being paid to the area of parametric methods in .the area of 
cost estimation in contracting. This effort will give greater 
insight into the potential uses for parametric cost estimating 
techniques. An analysis of the problems that may be 
encountered at the onset of policy implementation may lead to 
a concerted effort to meet them with corrective action before 
they become impediments in program management. The analysis 
provides information regarding how companies contracting with 
the Federal Government and Federal Government personnel 
interacting with these companies can effectively use 
parametric cost estimating techniques to support cost 
proposals submitted to the DoD. 

D . RESEARCH QUESTIONS 

Given the stated objectives for this thesis, the areas of 
research are; 

Primary research question: 

How could a DoD emphasis on using parametric methods in 
cost estimating affect the procurement process? 
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Subsidiary research questions include: 

1 . What changes could offerors make to their proposal 
preparation, submission, and support processes to 
accommodate using parametric methods, techniques, or 
software in estimating potential contract costs? 

2 . What impacts could emphasis on using cost estimates 
based on parametric estimating techniques have on source 
selection criteria and on the evaluation of proposals by 
the DOD? 

3. How could source selection criteria and evaluation 
factors be constructed to fairly evaluate a proposal that 
uses a parametric cost estimate with one that does not? 

4. What effects could using parametric cost estimations 
have on negotiations? 

E. RESEARCH METHODOLOGY 

The basis of the study is a literature review through the 
Defense Logistics Studies Information Exchange (DLSIE) , the 
Defense Technical Information Center (DTIC) , and various 
publications and journals. Data were also collected through 
survey questionnaires from Federal Government policy offices, 
acquisition management organizations, contracting offices, 
companies doing business with the Federal Government, and 
several academic institutions. The survey used is provided as 
Appendix C and the data from the survey are presented in 
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Appendix E. Personnel interviews were conducted with several 
representatives from DoD organizations, industry 
organizations, and companies doing business with the DoD. 
Interview questions are presented in Appendix D and the 
information from the interviews is incorporated in. the body of 
the text. 

F. ORGANIZATION OF THE STUDY 

Chapter I of this thesis is an introduction providing a 
brief description of the topic, the scope of the research 
effort, and the research methodology used. A review of the 
background of parametric cost estimating is discussed in 
Chapter II. Cost estimation techniques and their historical 
uses in Department of Defense contracting are also discussed. 
Chapter III describes the process of developing a parametric 
cost estimate. In Chapter IV, the data collection methodology 
is described, the data displayed, and data analysis presented. 
Chapter V concludes the research effort with a summary of the 
findings, conclusions, and recommendations. Appendices will 
present definitions and acronyms used and the survey vehicle 
used to collect the data. 
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II. BACKGROUND REVIEW AND HISTORICAL PERSPECTIVE 



A. PURPOSE 

The purpose of this chapter is to lay the historical 
foundation from which the study was conducted. Beginning with 
the development of parametric cost estimating techniques, the 
chapter next discusses their application in the source 
selection segment of the Federal Government' s acquisition 
process. Parametric cost estimating techniques are defined, 
as well as several other terms relating to the acquisition 
process, as they relate to the use of parametric cost 
estimating methods or techniques. Potential applications in 
both Federal Government acquisition and commercial arenas are 
examined. Finally, the chapter closes by discussing the 
advantages and limitations of using parametric cost estimating 
techniques. This information provides the background for 
understanding why there is an interest in applying the 
techniques more widely than they have been in the past. 

B. BACKGROUND 

1 . History 

Historically, the use of parametric cost estimates for 
pricing purposes is relatively new. Whereas price analysis 
is practiced in some form for every purchase that has been 
made, the techniques used in parametric cost estimating have 
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their origins in the learning curve theory espoused by T.P. 
Wright for use in the aircraft industry in the latter part of 
the 1930s. [Ref 37:p.4] The common sense notion that 
production of an item becomes more efficient as the quantity 
produced increased was finally quantified. Having this 
relationship mathematically derived rather than estimated 
through experience and guesswork, allowed manufacturers to 
begin forecasting requirements more accurately over several 
production runs. Managers could predict the cost of resources 
needed to produce or the delivery schedule for a given item at 
a given time in the production cycle through the use of a 
logarithmic graph of units of production and labor hours 
needed to produce a given unit of production. [Ref 37:p.4] 

Following World War II, other estimating techniques were 
developed and refined. Wessel, in his article, describes 
several methods in use at that time to rapidly estimate 
project costs for use in comparative analysis of capital 
investment opportunities. [Ref 46:p.l69] Of interest to the 
development of parametric cost estimating is the discussion of 
cost per ton of product in the U.S. chemical industry. 
Although this is certainly not the first time ratio analysis 
was used in industry, it is an indicator that the technique 
was in common use. Attempts were also being made at that time 
to increase the accuracy of the estimates through analyzing 
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the data and correlating costs to several production factors. 
At that time, the estimates were said to be accurate enough 
to produce results which allowed managers to narrow the choice 
of new products or processes to the most competitive 
available. 

Descriptions of several common estimating techniques used 
by commercial industry in the late 1950s are: 



1. Detailed Estimates are individual cost elements 
compiled from in-depth engineering projections. 

2. Preliminary-Bill Estimates are materials priced from 
price lists or quotations and labor costs derived from 
labor-material ratios seen on similar jobs in the past. 

3. Layout Estimates are equipment costs obtained from 
vendor quotations or price lists and then corrected for 
various inflation factors using appropriate indices. 
Material costs are estimated based on some cost per unit 
of output basis, with engineering costs calculated from 
a percentage curve based on past experience with similar 
projects . 

4. Equipment-Ratio Estimates are estimated by using 
ratios developed from experience on past projects. 

5. Capacity-Cost Estimates are estimates made by 
projecting costs for a new plant by using a logarithmic 
plot of similar plant capacities and costs from 
historical records. [Ref 14:p.l31] 



These techniques show increasing interest in producing 
accurate cost estimates and from that interest greater 
sophistication in generating those estimates. While there 
were elements of these techniques that are necessary for 
parametric cost estimating, they had not yet come together. 
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More importantly, they did lay the foundation for parametric 
cost estimating. To produce estimates using many of these 
methods the way data were collected, analyzed, displayed, and 
stored had to be changed. 

Concurrent with the interest in the U.S. commercial 
industry with greater estimate accuracy, the Federal 
Government, particularly the DoD, was under pressure to 
develop more accurate estimates. As in industry, the Rand 
Corporation noticed relationships between costs and production 
outputs. [Ref 37:p.7] Whereas industrial applications related 
costs to production quantities or consumption of raw materials 
used, the Rand Corporation analyzed the correlations between 
costs and variables that described operating characteristics 
of aircraft. In particular, it was observed that certain 
classes of aircraft had similar cost behaviors regarding 
weight, speed, range, and altitude. These behaviors were 
called Cost Estimating Relationships (CER) and are the basis 
for parametric cost estimating. The combination of learning 
curve theory, available data from extensive production of 
similar items using similar processes (aircraft production 
during and after World War II), and development of the idea of 
the CER came together in the form of parametric cost 
estimating. 
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The parametric estimation methods have been used for many 
years with ever widening applications being found for their 
use. From some of the original uses in the Department of the 
Navy (DoN) as first order estimates relating aircraft and ship 
weight and cost to develop simple and fairly reliable cost 
estimates, the methods have been widely applied. Parametric 
cost estimating allowed planners to make accurate estimates of 
system costs at early stages in design, a solution recognized 
in both commercial and Federal Government applications. The 
Secretary of Defense (SECDEF) became convinced of the utility 
of Independent Parametric Cost Estimates (IPCE) in 1971 as a 
tool to control cost growth in major weapon systems. [Ref 
25:p.l2] He issued directives to use the techniques in cost 
estimating at the program level to solve the problem of 
accurately estimating Research, Development, Test and 
Evaluation (RDT&E) and initial production run costs. 
Estimation difficulties in the early stages of production at 
that time were exacerbated by the rapidly escalating pace of 
technological development. It was felt that success in the 
early stages of a program's life would lead to reductions in 
program cost overruns. In conjunction with management 
difficulties caused by cost overruns, there was enormous 
pressure to reduce the DoD budget. To manage these budgetary 
pressures, SECDEF hoped to have each Service perform IPCEs at 
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each key decision point in the weapons acquisition process to 
increase the accuracy of cost estimates. 

Since the inception of the idea of parametric cost 
estimating and study by the Rand Corporation, there has been 
increasing application in commercial industry and. the Federal 
Government for planning purposes. Initial uses of parametric 
cost estimates centered mainly on [Ref 20:p.l2] 

1. Identifying possible cost/performance tradeoffs in the 
design effort. 

2. Providing a basis for cost/ef fectiveness reviews of 
performance specifications. 

3. Providing information useful in the ranking of 
competing alternatives. 

4. Suggesting a need for identifying and considering new 
alternatives . 

The reasonably accurate estimates in the early stages of 
program development and widespread applicability contributed 
greatly to the acceptance of parametric cost estimating as a 
valid method. Increasing competitive forces have encouraged 
their use in commercial applications while high level interest 
and greater public scrutiny have led to wider use in DoD 
areas . 

Figure 1 describes a general relationship between 
procurement cost estimating techniques used and the time line 
throughout a DoD-managed program. For comparison purposes, 
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the cost estimating techniques used before the advent of 
parametric cost estimating are shown along the base of the 
figure where they might have been used at various times during 
the program life. Parametric cost estimates are used in all 
phases of a program life cycle but are most often the basis of 
estimates in the Program Definition and Risk Reduction 
(PD&RR) (Phase I) of a program life cycle. [Ref 26:p.21] 
Parametric cost estimating is perceived as the best method of 
generating a detailed estimate from the available data when 
only mission and performance parameters have been described. 
[Ref 23:p.291] 

Phase O Phase I Phase II Phase III 




EO Expert Opinion P * PArametric A = Analogy BU « Bottoms Up LREP - Low Rate Initial 

Production 

Figure 1: Program Phases and Types of Estimates 
Source: Refs 22, 25, and 33 
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From its inception, parametric cost estimating has 
experienced an ever expanding base of uses. Aircraft programs 
have long been the beneficiaries of parametric price 
estimating techniques. Weight has been the primary 
independent variable to parametrically estimate . costs for 
aircraft systems. Analysis by the Rand Corporation showed 
that another useful variable that shows a good correlation to 
cost is speed. [Ref 20:p.21] Other analyses have defined a 
parametric relationship between cost and wetted area, aspect 
ratio, number of engineering drawings, number of parts, and 
even time in aircraft development and production programs. 
[Ref 20:p.34] Despite all of the analysis, weight is the most 
consistent and accurate parameter to estimate aircraft program 
costs. [Ref 20:p.36] No longer applied solely to Federal 
Government aviation programs, the techniques have been used in 
a wide variety of Federal Government areas. Projects in the 
Federal Government that have used parametric cost estimating 
techniques include Department of Energy construction of dams 
and mining structures. Commercial uses range from 
construction and aircraft manufacturing to computer software 
generation projects. 

The DoN is heavily dependent on the use of parametric 
pricing for major systems planning. It forms a major part of 
shipbuilding and aircraft cost estimating both at the 
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aggregate level (program) and equipment level. The techniques 
are used in projections in shipbuilding to estimate costs 
within each element of the Expanded Ship Work Breakdown 
Structure. Similar to the CERs observed from aircraft 
production, one of the most useful correlations is that of a 
ship's gross tonnage to cost. [Ref 35:p.23] One of a ship 
class's most consistent physical aspects that shows a useful 
correlation to cost is that of gross tonnage, making it a most 
useful variable in ship cost estimation. Other CERs that have 
been developed and used successfully for shipbuilding 
estimates are man-hours to gross tonnage and power rating 
ratios to gross tonnage relationships. [Ref 35:p.37] 

The development and use of parametric cost estimating has 
followed that of the development of computing capability. 
Prior to the widespread availability and use of computers, 
data collection and analysis on the scale required for 
parametric cost estimating was prohibitively expensive and 
time consuming. As computer hardware improved so has the use 
of parametric cost estimating. Software has also made 
tremendous improvements that parallel those of hardware. More 
sophisticated processing of data and user friendly programs 
have combined with the hardware improvements to bring 
parametric cost estimating into common use. These 
capabilities allow quantitative analysis of such accumulations 
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of data that, without the computer, parametric cost analysis 
would not be useful in terms of timeliness even if it were 
possible at all in a manual environment. 

2 . Source Selection Process Development 

Parametric cost estimating occupies an important niche in 
the source selection process. For the contractor proposing to 
do business with the Federal Government, estimate accuracy 
could mean the difference between winning and losing the award 
of the contract. A contractor who uses a poor estimate in a 
Firm Fixed-Price (FFP) contract environment will suffer by 
using that estimate. He will either lose money through 
underbidding his costs if he bids lower than his actual costs 
and wins the contract award, or he will lose business if his 
estimate is too high, he bids too high, and he is not awarded 
the contract. For the Federal Government, a reliance on a low 
estimate could lead to quality control problems. The 
contractor who won the contract with a low bid will attempt to 
limit his loss through contractor cost cutting measures. In 
a Cost-Reimbursement contract environment, additional 
management resources will be required of the Government to 
control cost overrun situations. The increased costs will 
also bring greater public scrutiny on both the Government and 
the contractor as cost overruns develop. In either case, 
inaccurate cost estimates damage the competitive industrial 



16 



base and have adverse impacts on the Federal Government 
procurement process. 

Along with the greater computing capacity, the growth of 
parametric cost estimating depended on the data that were 
available for analysis. Other Federal Government applications 
depended on the collection of data for analysis. This data 
collection was made available in large part through dependence 
on detailed engineering cost and schedule estimates. The 
Truth in Negotiations Act of 1962 (TINA) codified cost and 
pricing data requirements that were to be submitted in 
contract negotiations with the Federal Government. In meeting 
these requirements with detailed cost estimates, commercial 
industry built the databases required for effective parametric 
cost estimating. 

Cost Accounting Standards (CAS) has also had a 
significant impact on the use of parametric cost estimating 
methods. One of the basic premises of CAS is consistency in 
estimating, acciamulating and reporting costs by a company 
proposing to do business with the Federal Government. Any 
parametric model used by a company must be able to segregate 
estimates of costs to match the data collection system used in 
that company. This in turn should match the proposed work 
breakdown structure (WBS) upon which understanding of the 
contractor's ability to perform work delineated in the 
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statement of Work (SOW) depends. Failures in this area could 
make it difficult for a buying organization, the Defense 
Contract Audit Agency (DCAA) , or the Defense Contract 
Management Command (DCMC) to evaluate a company's cost 
proposal. Differences here could also result in CAS 
violations which in turn could lead to adverse effects on 
prior performance ratings. 

Cost estimating plays a major part of any procurement 
decision from influencing the budgetary processes to final 
source selection. The Competition in Contracting Act (CICA) 
of 1984 followed TINA, and both had tremendous impacts on the 
source selection process in Federal Government procurement. 
Where TINA determined data to be submitted as the basis for a 
cost proposal, CICA clearly allowed for negotiated procedures 
in a competitive environment for Federal Government 
procurement when sealed bidding conditions do not apply. [Ref 
41] Beyond the FAR and Defense Federal Acquisition Regulation 
Supplement (DFARS) guidance, executive agencies have published 
additional regulatory constraints that affect the conduct of 
Federal Government procurement actions. General guidance is 
given, however, in the DCAA Contract Audit Manual. [Ref 
37:Appendix C] Parametric cost estimating techniques are not 
specified for use or directed to be used in any of this 
guidance; rather their use has grown as a sensible. 
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businesslike answer to meet the needs of individual programs 
and procurement actions. [Ref 37:p.7,198] 

In acquisition planning, specified in CICA and depicted 
in Figure 1, parametric cost estimating is applied to provide 
a manager with useful data upon which he can make an informed 
decision. Source selection planning for a particular 
procurement is only a part of the overall plan, but it is the 
place where the uses of parametric cost estimating are best 
illustrated. An individual procurement action using 
parametric cost estimating can occur in any of the four phases 
shown in Figure 1 . Determining that paraimetric cost 
estimating is useful in the conduct of a procurement action 
may have the most beneficial outcome when used in the planning 
stages. Parametric cost estimating considerations can best 
be integrated into procurement planning to fully exploit their 
potential . 

Use of parametric cost estimating techniques could first 
impact the individual procurement by affecting the structure 
of the solicitation for an item or service. The buying 
activity may choose to construct the solicitation to accept, 
reject or encourage the use of parametric cost estimates. 
Requiring the use of parametric cost estimating techniques may 
in some instances restrict competition to those with the 
experience in not only providing the goods or services 
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required but also knowledge and experience with the techniques 
specified. Rejecting the use of parametric cost estimating 
techniques as valid methods of supporting a price proposal 
could remove a competitive advantage for a company in another 
situation. 

From the solicitation, the companies submitting proposals 
will construct their proposals in a way to present their 
company as the best value for the buying organization. They 
will present the data required by the solicitation in support 
of their proposal. For those companies familiar with 
parametric cost estimating techniques, this could be a direct 
competitive advantage. Conversely, for those who are not 
familiar with them it could be an administrative burden at 
best and a liability in the competitive environment if they 
are required by the buying agency. The buying organization 
should recognize the differences in how parametric cost 
estimates are developed from the traditional bottoms up 
estimates and structure the support data requirements 
accordingly. In either situation, the buying organization 
should restrict the data to be submitted to only those 
required to support analysis of the proposal. 

The evaluation criteria for the procurement, expressed in 
the solicitation, should drive the data collection efforts. 
These criteria are required to be explained in the body of the 
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solicitation to potential offerors. Companies proposing to do 
business with the Federal Government will use these criteria 
as considerations in the construction of their proposal. Each 
buying organization may have a different view of the 
interrelationships between the elements in the evaluation 
criteria. Parametric cost estimating could have impacts not 
only in the area of cost comparisons and cost realism but also 
in the technical proposal involving understanding the 
requirement. In the evaluation of management capability, 
parametric cost estimating could again demonstrate a company' s 
level of understanding of the requirement as well as affect a 
company's score on past performance. 

The proliferation of data collection systems, management 
systems, and user friendly statistical analysis programs make 
the techniques of parametric cost estimating more available 
than ever before. The techniques have generated an industry 
whose goal is to provide parametric estimating services and 
tools to potential users. The DoD-maintained Constructive 
Cost Model (COCOMO) was developed to estimate software cost 
estimates. One of the more widely used parametric cost 
estimating models produced by commercial industry is the 
Programmed Review of Information for Costing and Evaluation 
(PRICE) . [Ref 37:p.219] The PRICE model was initially 
developed by the RCA Corporation in 1974 to provide 
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engineering and manufacturing costs for equipment having a 
wide range of applications. It has undergone several 
enhancements since its inception and is now a family of cost 
estimating models for hardware and software projects. [Ref 
21:p.l45] PRICE, however, is only one of many models 
developed by industry and the Federal Government to meet the 
needs of program and procurement management. 

The development of parametric cost estimating has 
progressed along with increasing budgetary pressures, and 
technical and management improvements. The current state of 
the art in all of these areas has brought renewed interest in 
their use by the upper echelons of DoD. As a step beyond 
their earlier uses, the present drive is focused on reduction 
of contractor costs from onerous Federal Government reporting 
requirements and an intense desire to reduce equipment 
development cycle times . 

One way to reduce the burden associated with 
providing cost and pricing data is to permit 
greater use of parametric estimating techniques. 
Properly calibrated parametric techniques can 
accurately estimate costs while reducing bid and 
proposal costs and proposal cycle time and 
expediting the Government evaluation process. [Ref. 
42:p.l] 

To reach the goal set out in the above memorandum by the 
Under Secretary of Defense, the use of parametric pricing will 
have to become more widely accepted throughout the DoD 
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acquisition community as a cost estimation tool rather than 
viewed as a program management tool for cost estimates. 

C. DEFINITIONS AND CONCEPTS 

1 . Parametric Cost Estimating 

Parametric cost estimating involves the collection and 
organization of historical information, analysis through 
mathematical techniques, and relating this information to the 
work output that is being estimated. A parametric cost 
estimate can be defined as an estimate which predicts cost by 
means of explanatory variables such as performance 
characteristics, physical characteristics, and characteristics 
relevant to the development process, as derived from 

experience on logically related systems. [Ref 37:p.l72, 

22:p.293] Combinations of CERs and their relationships are 
used to describe the likely cost behavior of a project. The 
emphasis in parametric cost estimating is to "focus on the 
cost drivers, not the miscellaneous details" [Ref 37:’p.ll] 
that are common in other cost estimating techniques. 

2. Parametric Cost Model 

A parametric cost model is simply the group of CERs and 
logical relationships used together to produce cost estimates . 
They can range from common rules of thumb, such as costs per 
gallon, to elaborate computer models. [Ref 17:p.l2] A 
parametric cost model is one derived from the relationships 
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between data points in a historical database used to provide 
a top down cost estimate. The buyer is attempting to relate 
what he or she wants to buy in units of capability to the cost 
for it. This is a departure from the traditional industrial 
engineering (or bottoms up) approach of describing what a 
buyer is paying for through the use of detailed cost 
accumulation systems. 

Parametric price analysis is used at all stages of a 
major procurement from the initial estimates to developing 
estimates for costs for components later in production. 
Figure 1 is a representation of how parametric pricing is 
typically used in comparison with other cost estimating 
methods . 

3. Cost Estimating Relationships 

The CER is the key component in the development of a 
parametric cost estimating model. CERs are described as: 

...an algorithm relating the cost of an element to 
physical or functional characteristics of that cost 
element or a separate cost element; or relating the 
cost of one cost element to the cost of another 
cost element. [Ref 37:p.l65] 

This idea is rooted in the fact that costs in the production 
of an item are related in a quantifiable manner to its 
physical or performance characteristics. An example of this 
principle in the Contract Pricing Resource Guides (CPRG) 
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describes the procedure of developing a CER for building 
houses and relating the costs to such variables as the number 
of bathrooms, available living area, exterior wall surface, or 
roof surface area. [Ref l:p.D-31] 

CERs can be broken down into two categories, cost- to-cost 
(overhead rates) and cost to non-cost (cost per pound of 
thrust) . It is critical that the cost analyst understand the 
logical relationship between cost and any independent variable 
for the model to be used effectively. This understanding 
leads to knowing the limits of the model and potential effects 
of using it with a new database. 

4 . Mathematical Techniques Used in CER Development 
Regression analysis and multiple regression analysis are 
two statistical techniques that can be used to relate the cost 
(dependent variable) to a characteristic (independent 
variable) . [Ref 23:p.291] The aim of regression analysis, in 
the form of applying linear, curvilinear, logarithmic, or 
exponential techniques to the data, is to derive the equation 
of a curve that predicts how an independent variable will vary 
with respect to a given parameter or dependent variable. In 
the case of procurement, interest has been focused on the cost 
and schedule estimating aspects. When the parametric 
relationship has been described, it can be used to estimate 
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system costs by substituting different cost, performance, or 
other characteristics into the equation. 

A second mathematical method that can be used to derive 
a CER is the Least Squares Best Fit (LSBF) method. [Ref 
37:p.41] This algebraic method minimizes the sum of the 
squared deviations of the observed values of a variable and 
calculated values of that variable. In this method, a series 
of calculations involving the mean values of the dependent and 
independent values and the point-slope formula are used to 
arrive at a formula that describes the CER between the two 
variables . 

Apart from strict mathematical derivation of the 
relationships between two variables, plotting a series of data 
points and graphically determining the line of "best fit" can 
also be used to derive a CER. [Ref 37:p.41] This method 
requires plotting the data points and simply drawing a line 
through the data points leaving an equal number of points on 
either side of the line. The drawn line must follow the 
general trend of the data points and every data point must be 
considered. Any outlying data points must not be discarded 
but instead investigated to understand why the variation in 
the data occurred. The point-slope algebraic formula for a 
line can be used to derive the CER in this method if a linear 
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relationship is noted. Curvilinear regression is another 
method available to develop the CER. 

5. Statistical Measures of Accuracy 

The statistical measures of the accuracy of an estimate 
based on the two described mathematical methods can be 
described in terms of: 

1. The Standard Error of the Estimate (SSE) - the 
variance associated with the prediction made from 
the estimating equation. [Ref 25:p.l3] 

2. The Coefficient of Determination - the closeness 
of fit between two variables or how much one factor 
is affected by another. [Ref 37;p.46] 

With these measures, a manager has another tool to assess 
the applicability of the CER to a given program or system. 
Using regression analysis, the data are then analyzed for 
relationships. The smaller the SSE and the larger the 
Coefficient of Determination the more accurate the equation 
will estimate the costs that will be incurred. Examination of 
the estimates made by the equations over the data range is a 
critical step in the evaluation. Often there are breaks or 
discontinuities observed in the data. At these points the 
relationship between one independent variable and the 
dependent variable breaks down and another relationship 
between the dependent variable and a second independent 
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variable becomes a more reliable predictor. There may be 
several CERs for a given item. or system depending on the level 
of production or range of a given performance characteristic. 

6. Model Validity 

Comparability between the items or services upon which 
the database is founded and those to be estimated is an idea 
even more basic to parametric cost estimating than the CER. 
Hertling defines parametric cost estimating model validation 
as "the process of determining that a model is a reliable 
predictor of costs for the type of system being estimated." 
[Ref 17:p.l3] This is also an issue in the analog method of 
cost estimating. Analog cost estimating methods are simply 
comparisons between past costs observed on similar types of 
systems or equipments and new ones. Although there may be 
some difficulty in determining what are logically related 
systems or programs that can be compared, the techniques can 
be easily applied if such a relationship can be defined 
between an existing system or program and a projected one. As 
in the analog methods of cost estimating, the greater the 
difference between the programs, systems, or equipments that 
make up the database and the projected system, the less useful 
is the estimate produced. 

The validity of the historical data analyzed is one of 
the most important factors in its later applications. [Ref 
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37:p.l2] Data input to the model must be verifiable and 
accessible to the cost estimator. Data accuracy is critical 
and has more dimensions than just similarity between the old 
and new programs, systems, or equipments. Parametric cost 
estimates are least useful where there are few data points for 
development of the basis for the CERs or when the methods of 
performing a task change significantly. The comparability 
between a new system and existing ones whose costs make up the 
database used affect the model's accuracy. Accuracy increases 
when the methods used for production of like items or systems 
are very similar. An example of this is a system that has 
undergone upgrades, with the new item or system incorporating 
technological innovations or increased performance 
characteristics beyond those previously attained. [Ref 
23:p.293] 

7. Model Validation and Calibration 

One method of validating a parametric cost estimating 
model involves generating estimates with the model for 
comparison with actual costs incurred. [Ref 37:pp.l8, 176] 
The differences must be explained and understood for the model 
to be useful. Without that understanding, the model must be 
modified and analyzed until it does produce reliable and 
accurate estimates. 
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Calibration of a model is basically the relation of a 
model to a particular instance rather than a general set of 
events. This would be a consideration in cases where a 
company uses a standard commercial version of a software 
product or model for estimate generation rather than a company 
that developed a model based on their own historical data. 
[Ref 37:pp.l7, 162] 

8 . Work Breakdown Structure 

In the Federal Government acquisition process, the WBS is 
often used as a base upon which the technical requirements for 
a program, system, piece of equipment, or individual contract 
action are developed. [Ref 37:p.l77] Through a .hierarchical 
family tree structure detailing the end product, work elements 
to achieve production of the end product, and their 
relationships, an understanding of the processes involved in 
the accomplishment of an objective is achieved. Specif ically ; 



...the work breakdown structure plays a significant 
role in planning and assigning management and 
technical responsibilities; and monitoring and 
controlling the progress and status of (a) 
engineering efforts, (b) resource allocations, (c) 
cost estimates, (d) expenditures, and (e) cost and 
technical performance. [Ref 37:p.l78] 



Guidance regarding the use of the WBS is provided in MIL-HDBK- 
881. [Ref 37:p.l77] DoD policy is that MIL-HDBK-881B should 
be referenced for guidance only in appropriate solicitations 
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defined as those solicitations involving cost risk to the 
Government. The intent is to improve program and contract WBS 
development, provide guidance to the contractor in extending 
the program WBS and maintain the integrity of the WBS. 

9. Price and Cost Analysis 

Price and cost analysis are used to determine that fair 
and reasonable prices are paid for products in business as 
well as Federal Government procurement. [Ref 2:pp.253, 262] 

Price analysis acts to keep a focus on the value of an item, 
and parametric cost analysis with its focus on cost drivers is 
a clear application of that principle. Price analysis allows 
a buyer to compare a price against a standard as a yardstick 
to determine if an item is worth the cost. Price analysis 
techniques are generally faster and do not cost as much to 
perform as cost analysis, a detailed breakdown of costs for 
each cost element in a proposal, but price analysis cannot 
give an accurate picture of what will be paid for an item in 
all cases. Price analysis is principally performed using 
these four methods [Ref 2:p.253] : 

1. An analysis of competitive price proposals is 

conducted. 

2. A comparison with regulated, catalog, or market prices 

is performed. 

3. A comparison with historical prices is performed. 

4. An independent cost estimate is used. 
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Price analysis only provides estimates of those costs 
relative to the accuracy of the prediction. It does not 
provide exact price predictions about costs that will be 
incurred in an effort. 

Parametric cost estimating ties together the concepts of 
price analysis and cost analysis. It is one of the methods of 
comparing a price in the present with those historically paid 
for an item and is one of the four methods a buyer has 
available to perform price analysis. Parametric cost 
estimating combines the price analysis technique of 
referencing historical prices with the mathematical analysis 
inherent in improvement curve theory from cost analysis. [Ref. 
2:p.267] 

10. Cost Realism 

Cost realism is a key concept in the source selection 
process entailing imderstanding a cost proposal in detail and 
evaluating the accuracy of component costs. The purpose of 
cost realism is to give a source selection official greater 
assurance that the effort will cost what is estimated in the 
proposal when the work is complete. [Ref 6:p.3-l] The 
inference is that the more realistic the proposal the more 
likely the project will avoid a cost overrun situation. 
Specific information about how a cost proposal will be 
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evaluated should be included in the source selection 



evaluation plan for a given procurement action and published 
in applicable solicitation documents. In general, the Federal 
Government will assess cost realism based on the difference 
between the contract offer and an Independent Government Cost 
Estimate (IGCE) . Another method of determining cost realism 
is to compare the proposed costs with a recalculation of the 
costs by the buying command based on contractor-provided data, 
past performance, and information from DCAA or DCMC audits. 
[Ref 6:p.ll-3] 

D. PARAMETRIC COST ESTIMATING APPLICATIONS IN ACQUISITION 

From the original efforts made by the Rand Corporation to 
develop a method of accurately estimating costs of proposed 
new systems, the uses of parametric cost estimating have 
expanded. Parametric cost estimates have been and continue to 
be used to [Ref 37:p.71] 

1. Identify possible cost-performance tradeoffs 

2. Provide a basis for cost/effectiveness review of 

performance specifications 

3. Provide information useful in ranking competing 

alternatives 

4. Identify and consider new alternatives 

5. Develop target cost estimates for planning purposes 

6. Estimate new program costs 
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7. Make decisions during bid or proposal strategizing 

8. Cross check actual costs during program life cycle 
with projected cost estimates made during planning. 

All of these uses have been primarily in the program 
management area. In the later stages of a program, they are 
used in a secondary role to bottoms up estimating. In fact, 
the CPRG has the use of CERs in price analysis shown as a 
secondary comparison technique. Emphasis is now on using 
parametric cost estimates as a primary source of cost 
estimates on an individual contract action basis. The primary 
goal is to maximize efficiency and minimize complexity of the 
solicitation, the evaluation, and the selection decision 
process . 

E. ADVANTAGES OF AND LIMITATIONS IN USING PARAMETRIC COST 
ESTIMATING TECHNIQUES 

1 . Advantages 

Parametric pricing using CERs has the advantages of being 
quick and simple to apply once the relationship has been 
described adequately. [Ref 37:p.38] This is a result of the 
reduced need for cost data specific to the cost elements of 
the new item or system. The only data needed are the 
independent variable inputs to the equations to get an output. 
Efforts normally used to generate detailed cost estimates, 
that may be no more accurate than the parametric estimates. 
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can be allocated to other more profitable endeavors. Where a 
detailed estimate may not be possible because of the lack of 
data, a parametric cost estimate may be possible. 

Parametric cost estimating methods do not have to be 
isolated to use only in large scale applications at the 
program or even entire equipment level. [Ref 37:p.68] They 
can be applied at the component level for inclusion in another 
type of cost estimate as well as on the macro management 
level. They provide a relatively quick and accurate way to 
produce cost estimates on a program or system level to be used 
as a sanity check. On the larger scale, when used to estimate 
costs at the system or program level, the data base includes 
costs incurred as a result of setbacks or other problems. 
With the database built on a continuing series of changes, 
usually in high technology areas, the model takes advancing 
technology automatically into account. Thus the CERs have a 
built-in correction factor for problem areas often observed as 
well as for program cost escalations and inflationary 
pressures . 

The accuracy of parametric cost estimates has improved 
significantly since their inception. [Ref 25:p.5] Using 
proven databases parametric cost estimating models are 
outstanding predictors of accurate estimates. From order of 
magnitude estimates in the early chemical industry 
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applications, to being equally as accurate as detailed 
engineering cost estimates in 1975 [Ref 25:p.6], the accuracy 
of parametric cost estimates has improved to the present 
claims of five percent difference from actual observed costs 
in some uses of the PRICE models. [Ref 21:p.l47] As in any 
modeling situation, accuracy of the inputs is a critical 
factor in obtaining useful and accurate output data. 
Parametric cost estimate accuracy is highly dependent on the 
following variables [Ref 14:p.l29]: 

1. Degree of project definition 

2. Comparability between projects in the database and the 

proposed project 

3. Status of model calibration and validation 

4. Data availability and depth of the database used 

5. Adjustments made to the data to ensure comparability 

within the database 

Another factor to consider is the increasing accuracy levels 
achievable throughout the life of a project. As more is known 
about a project, more accurate estimates can be made over 
time. The benefit of a parametric cost estimating system is 
the speed at which revised estimates can be generated as new 
information becomes available. The flexibility inherent in 
the parametric cost estimating methods is also demonstrated in 
their uses in conducting "what if" analyses. 
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2 . 



Limitations 



Despite all of the advantages, there are some 
difficulties with using parametric cost estimating methods. 
They may not be applicable in all cases because one may not be 
able to find a close relationship between costs and any 
independent variable that can be easily measured or 
quantified. In some cases the models use CERs that do not 
provide sufficient detail to accurately predict costs. To 
maintain accuracy, the historical data base must be updated 
continuously, or the equations may not accurately represent 
the relationship between costs and the independent variable 
chosen as a basis for estimation. Technological changes in 
manufacturing processes, to the equipment itself, or 
significant changes in the management structure of an 
organization may make it to difficult to compare and correlate 
with the established database. [Ref 14:p.l30] 

Additionally, several problems distort the results from 
parametric price estimation. Aside from inaccuracies that can 
be introduced by design into the equations to allow greater 
applicability, the size of the data base can have dramatic 
effects on the accuracy of estimates. [Ref 14:p.l32] Larger 
data bases will generally have greater accuracies than smaller 
ones as the statistical models increase in accuracy. Data 
point dispersion will also affect the accuracy but this can be 
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measured through the use of SSE and coefficient of 
determination values. The standardization and normalization 
of data that is required to be able to compare the results 
across programs can also adversely affect the output of a 
parametric cost estimating model. 

Two such areas of normalization are in the adjustments 
for inflation by Price Level Adjustments and Cost to Quantity 
Adjustments. [Ref 16:p.l20] The data to be analyzed must be 
adjusted to a base year constant dollar level for comparison 
purposes through the use of standard indices. To compare 
across different procurement sizes in terms of quantities of 
units purchased, the data used in analysis must also be 
adjusted to develop consistent cost measures. The most common 
method used to do this is through the use of learning curve 
and average cost per unit calculations and techniques. [Ref 
37:p.l9] 

As accurate as parametric cost estimating is, it cannot 
give a 100% accurate picture of what will be paid for an item. 
The nature of estimating is to predict costs in the future 
based on extrapolating known data forward in a new situation. 
The only way to obtain 100% accuracy is to have full 
visibility of costs throughout the life of the project and 
total them at project completion. 
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Development of a parametric cost estimating model and its 
maintenance is expensive. Database management itself is 
costly as is maintaining a system of data collection to build 
and update the information required to keep a parametric cost 
estimating model up to date. Without constant maintenance, 
CERs and the model that is based on them can fall out of date 
rapidly. Finally, although parametric cost estimating models 
are very flexible within the boundaries in which they were 
derived, the degree of difficulty in redefining those 
boundaries makes them inflexible in accommodating major 
program changes . 

F. SUMMARY 

Parametric cost estimating has enjoyed a long history and 
it is used in numerous applications. The foundation of 
parametric estimating system is the integrity of the 
historical database and the CERs that make up the model. 
Before any parametric model is used, the model and database 
must be examined to ensure that they are up to date and 
accurate. The model should be calibrated for a specific 
situation and then validated before it is relied upon. With 
this information as a base, the following chapter will use the 
concepts addressed in this chapter to develop a cost 
estimating model using parametric methods. 
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III. DEVELOPING A PARAMETRIC COST ESTIMATING MODEL 



A. PURPOSE 

This chapter will describe a process by which a 
parametric cost estimating model might be developed. Knowing 
how a model was developed allows the user to understand the 
strengths and limitations of the model. This in turn will 
lead the user in the most beneficial application of the model 
to the data available or give him the knowledge he will need 
to optimize the model to meet his specific needs. 

There are eight basic steps in the construction of a 
parametric cost estimating model. [Ref 25:pp.8-35] 



1 . Gather data on both costs and potential independent 
variables such as speed, weight, or number of lines of 
computer code . 

2. Plot the data. Through a simple visual analysis, the 
user may determine that there is no useful correlation 
between cost and the independent variable. 

3. Mathematically or graphically relate the data points 
is the process of CER formulation. 

4. Examine the CER. Does the CER logically apply to the 
data? 

5. Refine the CER and review the data. 

6. Test the CER. Does the CER predict costs comparably 
to known historical costs for the variables used in the 
CER? 

7. Combine the CERs in the Parametric Cost Estimating 
Model . 
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8. Test the Parametric Cost Estimating Model. Does the 
model predict costs comparably to known historical costs? 

For the purposes of this effort in model construction, a study 

by Raymond E. Moore III in 1975, Cost Estimating Relationships 

For Naval Surface Ship Electronic Warfare Equipment , will form 

the basis. [Ref 25] In that study, Moore developed a 

parametric cost estimating model for electronic equipment 

based on a review of the costs and technical data for 12 

existing systems. These data are shown in Table 1. 



Table 1 

Electronic Warfare Equipment Data Matrix 



System 


Unit 

Cost 


Year 


Developm 
ent Cost 


Instal 

lation 

Cost 


Weight 


Volume 


Sensitiv 

ity 


Power 

Usage 


Quantity 


Gain 


SLQ-19 


758 


69 


1886 


268.1 


5190 


275.2 


-35 


15.0 


4 


0 


SLQ-26 


1572 


70 


3654 


582.5 


6016 


262.1 


-36 


15.0 


8 


15 


SLQ-28 


4092 


71 


11694 


562.5 


21032 


560.7 


-20 


15.0 


5 


15 


WLR-11 


120 


72 


360 


245.3 


375 


16 


-65 




12 


0 


SLQ-30 


408 


74 


1600 


37.0 


2060 


131.6 


-45 


1.0 


100 


15 


SLQ-12 


250 


65 


1754 


52.1 


2614 


154.2 


-50 


.4 


50 


20 


SLQ-17 


1625 


73 


7256 


168.9 


6000 


450 


-48 


15.0 


30 


15 


SLR- 12 


6.8 


65 


380 


15 


200 


12 


-60 




315 


18 


WLR-1 


30.0 


63 


3598 


280.1 


1958 


185 


-80 




600 


15 


WLR-3 


3.0 


63 


45 




25 


1 


-45 




400 


15 


WLR-8 


1200 


73 


11694 




2908 


220 


-85 




7 


15 



Source: Reference 24, p. 40 
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B . PLOTTING DATA 

Moore compiled these data and analyzed the relationships 
between both development and installation costs and all of the 
parameters listed in Table 1. [Ref 25 :pp. 11-21] For 
illustration purposes, this study will demonstrate the second 
step in the development cycle by plotting the system weight 
against the development costs in Figure 2. 



Electronic Equipment 

Weight Vs. Cost 




0 5000 10000 15000 20000 25000 

Weight 

Figure 2: Weight/Cost Data Plotted Graphically 
Source: Developed by researcher. 

At this point, a visual inspection shows that although 
there are some outlying data points, it appears as if a useful 
mathematical relationship can be derived from the points. In 
contrast. Figure 3 is a graph of system development costs 
plotted against quantities of the systems acquired. From 
visual inspection of the graph there appears to be no useful 
correlation between these two variables. 
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■ 

Electronic Equipment 

Quantity Vs. Cost 




0 100 200 300 400 500 600 

Quantity Acquired 



Figure 3; Quantity/Cost Plotted Graphically 
Source: Developed by researcher. 

To develop a CER to be used in parametric estimation, the 
buyer has to first decide what he wants to estimate (cost, 
schedule, material, labor, or some other parameter) and then 
determine how the variables are related in the work to be 
performed. Will the price estimate be derived directly from 
the data or tied to a variable to which a cost is then 
applied? Here, it has been determined that cost will be the 
dependent variable to be derived directly from the independent 
variable, weight. 
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C. CER DEVELOPMENT 

Having selected the characteristic that will give an 
accurate representation of the relationship between the 
independent and dependent variables, the relationship must be 
quantified. The independent and dependent variables must be 
not only related in some manner, but they must be related in 
a definable way. There has to be an understandable 
relationship between the data points, and the data points must 
be easily measurable or inferred from available data. 

Using linear regression techniques a line can be used to 
approximate the relationship between the data points. Here 
the relationship between development cost and weight can be 
described as Y= (.950082 x System Weight) + 604.4864. This 
CER is shown in Figure 4. 



Electronic Equipment 

Weight Vs. Cost 




Y=550082(Wei|}tt) + 604.4S64 



Figure 4: CostAVeight CER Overlaid on Data Graph 
Source: Developed by researcher. 
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The coefficient of determination for this CER is 



calculated to be .691395 and the SSE to be 4519.926. A 
coefficient of determination with a value of 1.0 indicates a 
perfect correlation between the two variables. In this case, 
for illustration purposes, the coefficient of determination is 
close enough to a value of 1 to be useful as an indicator of 
future costs. The SSE is relatively large, in part due to the 
wide range of development cost values used in the computation 
and the small database used. 

D. COST ESTIMATING RELATIONSHIP REVIEW/REFINEMENT 

A test of the CER developed here shows that it does make 
intuitive sense. As one would expect, costs appear to rise as 
the weight of the system increases. An examination of the 
relationship and the data supporting it are now made in an 
attempt to increase the accuracy of the CER. Should some of 
the outlying data points be discarded and is the mathematics 
used logically sound are two questions that have to be 
answered each time the CER is used. As an illustration to 
show the effects of data manipulation, two data points will be 
discarded and the results shown graphically in Figure 5. 

Deleting the two data points from the database has a 
great effect on the interpretation of the information. The 
CER developed has changed to Y= .834494 (Weight) + 1300.296 
with a coefficient of determination of .777718 and a SSE of 
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1643.74. While this action has increased the accuracy of the 
CER, it may have also reduced the effective range over which 
it is valid. Any use of the two CERs developed must take 
these data manipulations into account. 



Electronic Equipment 

Weight vs Cost 




0 1000 2000 3000 4000 5000 6000 7000 

Weight 




Figure 5: Revised Data and CER 



Source: Developed by researcher. 



E. COMBINING CERS INTO PARAMETRIC COST ESTIMATING MODELS 

Concluding the data manipulation and CER refinement 
initiates the final step in the process of CER development. 
In this construction, historical costs are compared with costs 
that would be projected by the CER using the known weights for 
the system. Table 2 presents those data for review. Here, it 
will be assumed that the CER data compares favorably with the 
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historical data and that they fall within an acceptable range 
for estimating costs. 



Table 2 

Actual vs. CER Projected Costs 



{ 

1 System 

[ 

! 


Weight 


Development 

Cost 


CER Projected 
Cost 


1 SLQ-19 


5190 


1886 


5535 


I SLQ-26 


6016 


3654 


6320 


I WLR-1 1 


375 


360 


961 


I SLQ-30 


2060 


1600 


2562 


I SLQ-12 


2614 


1754 


3088 


I SLQ-17 i 


6000 

j ... 


7256 


6305 


I SLR-12 I 


200 


380 


795 


I WLR-1 


1958 


3598 


2465 


i WLR-3 i 

1 1 


25 


45 


628 



Source: Developed by researcher. 

Up to this point, the construction has been concerned 
with developing a single CER for estimating development costs . 
The purpose of the effort, however, is to derive a parametric 
cost estimating model for a new electronic system. 
Development costs comprise only a part of the costs of any 
system. Other costs that might be considered are installation 
costs, spare parts, and training for operators. To build a 
parametric cost estimating model, all of these factors must be 
combined in some form. They may be estimated using a CER, as 
were the development costs, or by any of the other cost 
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estimating techniques available. A parametric cost estimating 
model considering these and other factors developed by Moore 
for this data set can be described by the following equation 
[Ref 23:p.21] : 

PROC = 2811.59351 + .05553 (DEVC) + .02824 (Weight) + 3.81016 
(Volume) + 25.98703 (Sensitivity) + 73.68718 (Power Usage) + 
77.15399 (Gain) - 1667.51343 (ACT) - 1027.33765 (LG) - 

1718.32324 (SM) 

which had the following statistics: 

Coefficient of Determination= .9997 
Standard Error of the Estimate =65.119 

where : 

PROC - Procurement Cost DEVC - Development Cost 

ACT - Active Equipment Code LG - Large Vessel Code 

SM - Small Vessel Code 

With the parametric cost estimating model complete, there 
is only testing against historical costs to verify the 
accuracy of the model remaining. The process described above 
is illustrative of one method available to develop a 
parametric cost estimating model. Using the parametric cost 
estimating model developed, the methods can be used in any of 
a program's life cycle stages for any number of estimating 
purposes . 
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F. SUMMARY 

This chapter developed a CER using regression analysis to 
demonstrate the relationship between the database used and the 
basis for the estimate. From this and other CERs particular 
to the equipment that is to be developed, a model would be 
constructed. This model would then be calibrated if that 
action was required and finally validated. After that point, 
it could be used to estimate the costs of electronic systems 
of a similar nature to the ones that made up the database. 
The next chapter describes the data collection and analysis of 
the data from surveys and interviews. 
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IV. METHODOLOGY AND DEMOGRAPHICS 



A. PURPOSE 

This chapter will discuss the data collection methods 
used in the study. Construction of the survey vehicle, 
Appendix C, and interview questionnaire. Appendix D, will be 
described followed by a discussion of the methods used to 
disseminate the survey. Data in this chapter will be 
presented as aggregate information describing the demographic 
breakdown of the respondents. 

B. DATA COLLECTION 

Data collection was centered around the- primary and 
secondary research questions. Information regarding the area 
of interest was collected throughout the use of a self- 
administering survey and personal interviews. The research 
questions formed the backbone of the survey, breaking the 
survey into discrete areas of interest. The survey was 
structured to follow a possible pattern of use for parametric 
cost estimating techniques through an acquisition cycle. Each 
major question is broken into several subsets to allow greater 
analysis of the topic in question. 

Questions one through four form the basis of the 
discussion of the use of parametric cost estimating in the 
procurement process. Ascertaining the risks inherent in the 
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program, system or equipment, its similarity to existing 
efforts, and the development of the databases available for 
analysis are key elements in determining how parametric cost 
estimating can be used in acquisition planning. Questions one 
through four represent some of the management areas to which 
parametric cost estimating have been and can be applied within 
the acquisition cycle. 

From the basis formed by questions one through four, the 
survey moves on to obtain opinions regarding the subsidiary 
research question: What changes will offerors make to their 
proposal preparation, submission, and support processes to 
accommodate parametric methods in estimating contract costs? 
Question five transitions from the general uses of parametric 
cost estimating methods to issues specific to the conduct of 
pre-award contract actions. Question seven continues to 
gather opinions regarding the question of offeror proposal 
preparation. The buyer's estimation of his position regarding 
the purchase of an item will be based on his understanding of 
the risks in the acquisition cycle and will be reflected in 
the solicitation he develops. 

The structure of a procurement action is in large part 
affected by its place in the acquisition cycle of a program 
and the overall acquisition strategy. In turn, an offeror's 
proposal is driven by the solicitation to which it is 
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responding. Elements of question six and question eight 
address the subsidiary research question: What impacts will 
emphasis on parametric cost estimates have on source selection 
criteria and on the evaluation of proposals by the DoD? The 
remaining elements of question six and question nine induce 
the survey respondent to consider the manner in which a 
solicitation is conducted. 

A central tenet in the Federal Government acquisition 
process is the fairness of the process. These questions 
collect data relating to the subsidiary research question: How 
will source selection criteria and evaluation factors be 
constructed to fairly evaluate a proposal that uses a 
parametric cost estimate with one that does not? The final 
area addressed in the survey is the negotiation process. 
Question ten collects opinions to answer the question: What 
effects will using parametric cost estimating have on 
negotiations? An additional page was attached for comments 
regarding the questions or any other concerns expressed by the 
respondents. These additional data have been incorporated 
into the analysis where applicable. 

In each of the questions in the survey, the respondents 
are asked to record their opinions about the use of parametric 
cost estimating methods using a numerical scale. Five point 
numerical scales were selected to provide a consistent, simple 
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method of coding the responses for data reduction and 
analysis. Two different scales were used in the survey. 
Questions one through four used a scale describing the utility 
of parametric cost estimating methods and questions five 
through ten used a scale describing agreement with the 
statements that make up the elements of those questions. 
Demographic data were also collected in an attempt to describe 
the population of respondents. The data were examined for 
relationships between subgroups represented in the survey 
respondent population and responses for particular questions. 
Through these methods the data were reviewed, analyzed, and 
related to the research questions. 

An interview questionnaire. Appendix D, was developed in 
the same manner as the survey instrument. The basic and 
subsidiary research questions were addressed in the interview 
questionnaire. The interview questionnaire was presented to 
seven individuals in personal interviews at the Naval 
Postgraduate School over the course of the research. 
Additionally, an initial form of the interview questionnaire 
was given to six people via telephone and E-mail at the start 
of the research effort. The information provided has been 
inserted in the discussion and background areas preceding this 
chapter as well as in the analysis of the survey data where 
applicable . 
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The survey was distributed as widely as possible to 
attempt to include as many elements of the DoD acquisition 
community as possible. The bulk of the surveys were 
distributed by mail using Federal Government agency mailing 
lists and organizational information. Electronic mail and 
facsimile transmissions were also used to distribute 
approximately 25% of the surveys. Potential survey 
participants were also identified through INTERNET searches 
and referrals during interviews. Through the survey's wide 
distribution across the DoD acquisition community, an 
understanding of the potential for the uses of parametric cost 
estimating methods was generated. Appendix E is. a collection 
of the data derived from the survey effort. 

An important group solicited for input were the members 
of the Federal Government's Parametric Estimating Initiative 
Steering Committee. This group was identified following the 
preface in Reference 36 and represents a cross section of the 
DoD acquisition community. The Steering Committee came 
together to identify methods of promoting the uses of 
parametric cost estimating techniques throughout the DoD 
community. For purposes of this survey, they responded to a 
pilot survey with recommendations and suggestions for 
improvement to the survey structure, direction, and the 
wording of questions. Their answers to the questions were 
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also most valuable in that they formed a basis for comparison 
against which the other demographic groups could be compared. 
Resident in this group is the most current information about 
the uses of parametric cost estimating across the entire 
spectrum of actions inherent in the DoD acquisition process. 
The Steering Committee is referred to as the control group 
throughout the analysis portion of this chapter. 

C . DEMOGRAPHIC DATA 

The survey distribution was tracked through the employer 
question of the survey. The industry and Federal Government 
employer categories were all-encompassing, mutually exclusive 
groupings that were easily tracked. Using this ■ breakdown as 
a basis, 1260 surveys were distributed over the period 21 
August 1996 to 15 October 1996. Throughout the research 
effort, it was found that many addresses were incorrect. It 
was assumed that the incorrect addresses resulted from base 
closures and realignments or agency reorganization efforts. 
Where possible, correct addresses were obtained and the survey 
redistributed. Figure 6 represents the survey distribution 
throughout the data collection effort. 
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Survey Distribution 

Employer Groups 




DCMC (14.45%) 
industry (2.21%) 

DOE (3.91%) 

OSD (0.78%) 

NASA (2.78%) 

USN (8.54%) 

GSA (0.64%) 

USA (23.63%) 



DCAA (37.37%) 



USAF (5.69%) 



Figure 6: Research Survey Distribution 



Source: Developed by researcher. 

The majority of the surveys were sent . to Federal 
Government contract support activities, DCMC and DCAA (52%) . 
It was felt that each organization interacting with a major 
portion of the DoD acquisition community across the entire 
spectrum of procurement actions would represent the DoD 
community experience with the uses of parametric cost 
estimating methods. 

A low survey return rate was observed from the data 
collected from 210 surveys returned. Only 16.5% of the 
surveys distributed were returned with data to be input for 
analysis. No opinion or no response to one or more questions 
was recorded on 47% of the surveys returned. Figure 7 shows 
the distribution of the returned surveys. 
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Group Return Rates 




n % Surveys Returned 



Returned Surveys 




DCAA 23.06% 



Industry d.61% 



NASA 4.33% 

U.$. Navy 10.56% 



OSD 0.96% 
DOE 0.96% 
6SA 0.46% 
USAF 6.18% 



DCMO 16.83% 



U.S. Army 24.99% 



Figure 7 : Survey Return Rates 
Source: Developed By Researcher 
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The percentages in the Returned Surveys chart of Figure 
7 display the percent of the surveys returned from an employer 
group. The Group Return Rate chart of Figure 7 shows the 
percentage of the surveys sent to each group that were 
returned by members of those targeted employer groups. The 
percentages of the surveys returned by groups ranged between 
68% from industry participants to a low of 3.64% by Department 
of Energy (DoE) personnel, both small groups within the survey 
population. Despite the relatively small number of the 
industry participants targeted (2%) , their high return rate 
made them the fifth largest group contributing to the 
database. The industry group may have had a disproportionate 
affect on the response averages because of this high return 
rate. The remaining groups had fairly similar return rates 
within a much smaller range of 12% to 25% and formed the 
majority of the data reviewed. 

As the survey was designed to ensure anonymity of the 
respondent, no effort was made to track response rates by 
organization within an agency, individually, or by any other 
demographic breakdown other than that shown above . The 
researcher was contacted on an individual basis, however, by 
approximately three percent of the survey population who 
indicated that they would not be participating. All of these 
respondents indicated that they were unfamiliar with the topic 
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and did not feel qualified to join in the survey. While 
parametric cost estimating methods have been in use within DoD 
for many years, their use has typically been isolated to upper 
and middle management areas dealing with major programs. Their 
use has not been as widespread in the other areas of DoD 
procurement that make up the bulk of the procurement 
transactions. [Ref 45:pp.7-ll] It is likely that a lack of 
experience with parametric cost estimating methods is a major 
factor in the survey's low return rate. 

The timing of the survey may also have had a negative 
impact on the return rate observed. The survey was 
distributed at the end of the Federal Government's fiscal 
year, typically one of the most active times of the year in 
the DoD acquisition community. Although the length of the 
survey was minimized to encourage participation, the survey 
had 55 individual data elements to be considered. Many of the 
recipients may not have had time to complete the survey within 
the requested time frame and thus chose not to participate. 

Demographic data were requested in four areas beyond the 
employer classification previously discussed. Participants 
were asked to identify themselves by their job classification, 
the length of time that they had been working in this area, 
how they categorized their work, and their level of education 
in mathematics. Each question was examined with these 
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categories in mind to identify trends, correlations, or points 
of interest in the data collected. In terms of job 
classification, three categories comprised the majority of the 
survey respondents: those identifying themselves as Cost or 
Price Analysts (26%), Contracting Officers (23%), and Auditors 
(22%) . Small numbers of participants identified themselves in 
the remaining job classification groups. 

Two employer groups were very homogeneous in composition: 
the DCMC and DCAA participants. The DCMC group identified 
themselves almost entirely within the job classification of 
contracting officer. The DCMC personnel made up 70% of the 
responding individuals of those identifying themselves as 
contracting officers. Unlike the DCMC participants, the DCAA 
group makes up almost all of the auditor job classification 
respondents. The DCAA and Auditor groups are nearly 
synonymous, with 96% of Auditors also classifying themselves 
as working for DCAA. All of the other groups were made up of 
combinations of the job classifications. 

The majority, 49%, of the participants identified 
themselves within the acquisition community and only 19% in 
the cost estimating community. A large portion of the 
respondents did not chose to respond to this question, perhaps 
overlooking it. There was no observed correlation between 
either community and any other demographic category similar to 
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that observed in the auditor and contracting officer job 
classifications. The majority of the respondents were very 
experienced in their positions, having more than 10 years of 
experience in either the acquisition or cost estimating 
professions. Very few people, 11% of the 210 participants, 
had less than 10 years experience. Also of note, only 39% of 
the respondents chose to provide their GS/SES level or 
military rank. The resulting analyses from these groups are 
limited because of their small size. 

The self-described work classifications are unlike the 
other demographic breakdowns in that these groups are not 
mutually exclusive. In question four, respondents were asked 
to select any of the work functions listed that they 
performed. They were not to limit their response to any 
single work function that made up the majority of their work 
effort. Reviewing or auditing estimates was chosen to 
describe the work most often performed by the respondents; 103 
selected this work description. This group' s answers roughly 
correspond to those in the job classifications of cost and 
price analyst and auditor. Negotiation preparation and 
proposal analysis was also selected by a large number of 
participants, 101 of 210, to describe their work efforts. 
Here, there is a correlation between this group of responses 
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and those of the contracting officer, contract negotiator, and 
auditor job classifications. 

Within the database, the majority, 88%, of the 
respondents answered that they had some degree of college 
mathematics education. A significant number of the 
participants had either college level algebra or business 
calculus, 30% and 33% of the respondents respectively. Of 
those identifying themselves as having taken an engineering 
calculus series, only 30% also identified themselves as 
engineers. The remaining individuals identified were spread 
across the rest of the job classifications as were the people 
who answered that they had either advanced degrees or majored 
or minored in mathematics while a college undergraduate. 

After reviewing the basic statistics for the entire 
database, the database was subdivided and analyzed. The 
information presented above and the demographic description in 
Appendix E were used to break the database into separate 
components to be compared against each other and the entire 
database statistics. In the following Chapter the comparisons 
made and analyses performed on the data will be described. 
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V, APPLICATIONS IN PROGRAM MANAGEMENT FUNCTIONS 



A. PURPOSE 

This chapter and the following two chapters will discuss 
the data analysis methods used and results observed in the 
study. Chapter V analyzes results from question one data 
related to parametric cost estimating applications in the 
broad scope of program management functions. Data pertinent 
to survey question one will be presented, related to the 
research question, and analyzed. Questions one through four 
used a five point scale that ranged from "vital to the 
procurement process," a one on the scale, to "a- hindrance to 
the procurement process," a five on the scale. Each question 
was further broken into smaller areas of specific interest. 

The data from each question were analyzed and the 
following statistics derived: 

1. The number of responses in each of the five 

categories . 

2. An average value of the responses. 

3. The standard deviation for the responses. 

4. The variation of the responses. 

5. The median value of the responses. 

6. The skewness of the responses. 
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These data are presented in Appendix E for the entire group of 
responses as well as for each demographic group. 

Analysis of all the questions and their components was 
conducted in a similar manner. Descriptive statistics for the 
entire group are presented, followed by group breakdown 
information, and the data presentation closes with analysis of 
variance and regression information. These data are presented 
in tables following an introductory paragraph. Group data for 
each question was chosen to be displayed for one of three 
reasons : 

1 . Some degree of consensus was observed in the group 
data analysis 

2. The group appeared as a point of interest in the 
analysis of variance, or 

3. The group data are illustrative of a point of interest 
found during the regression analysis performed. 

In both the regression analyses and analyses of variance, a 
95% confidence interval was used. For simplicity, group 
descriptive statistics for the first subgroup in the first 
question, describing the utility of parametric cost estimating 
in the area of benchmarking process costs, will be presented 
in tabular and text format with the data for the remaining 
questions shown in tabular format only. An analysis of the 
displayed data will follow those data for each question. 
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B. BROAD SCOPE UTILITY 

The first question asked the respondent to provide an 
opinion regarding the utility of parametric cost estimating 
methods in several functional areas. This question was 
divided into 12 segments, five dealing with . overarching 
functions common to the acquisition cycle in general and seven 
treating procurement of specific items or services. 

1 . Benchmarking Process Costs 

Benchmarking process cost was the initial subgroup within 
question 1 . Parametric cost estimating methods may have 
applications in situations where new processes are being 
developed. The methods may be most useful • where these 

processes are sufficiently similar to past or existing 
processes and the new processes are expected to have similar 
cost characteristics or patterns. As a group, the 
respondents' opinions indicated that they thought parametric 
cost estimating methods were somewhat useful in this 
functional area. The largest single group, 56 respondents, 
selected somewhat useful, a three on the scale, as describing 
the methods' degree of utility here. Only one respondent 
thought that the methods could hinder cost estimating if used 
here and 21 people thought they were of no use. 

The corporate policy group selected either vital or very 
useful to describe the utility of parametric cost estimating 
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methods in this situation. This agreement is statistically 
demonstrated in the small standard deviation of .548 for the 
group and a mean value of 1.4 for all of the responses. These 
statistics must be tempered, however, with the group size. 
With only five individuals, it is a very small. part of the 
database of 210. Other small groups had similar occurrences 
in the data analysis. The program staff group, five persons, 
three industry cost analysts and the U.S. Army cost analyst 
group, a group of nine, were unanimous in their belief that 
parametric cost estimating methods were somewhat useful for 
benchmarking process costs. U.S. Air Force cost analysts, 
another small group of five, agreed with the cost, analyst from 
the General Services Administration (GSA) that the methods 
were very useful for this purpose. These data described are 
displayed in Table 3. Analyses of variance and regression 
analyses pertinent to question one and discussions of these 
results follow. No other trends or indications were observed 
from the group statistics analyzed. 

Other analyses performed with the data from this question 
were analysis of variance and regression analysis. In 
reviewing the results shown in Table 4, a relationship was 
noted between job classification and the responses for 
benchmarking process costs. A p value of .04 indicates that 
a relationship exists between the responses in the job 
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classification group and their answers for benchmarking 



process costs. 



Table 3 

Benchmarking Process Costs Group Data 



Group 

Description 


Min 


Max 




Median 


a 


Q1 


Q3 


Corporate 

Policy 


1.00 


2.00 


1.40 


1.00 


.55 


1.00 


2.00 


Program 

Staff 


3.00 


3.00 


3.00 


3.00 


.00 


3.00 


3.00 


Industry 

Cost 

Analysts 


3.00 


3.00 


3.00 


3.00 


.00 


3.00 


3.00 


USA Cost 
Analysts 


3.00 


3.00 


3.00 


3.00 


.00 


3.00 


3.00 


USAF Cost 
Analysts 


2.00 


2.00 


2.00 


2.00 


.00 


,2.00 


2.00 


GSA Cost 
Analysts 


3.00 


3.00 


3.00 


3.00 


.00 


3.00 


3.00 



Source: Developed by researcher. 



A point of interest in this analysis is in the difference 
of opinion between the program managers and other job 
classification groups. The six program managers who chose to 
respond to this question believed that the methods were vital 
or very useful in benchmarking process costs. The other 
groups felt that the methods were either very useful or only 
somewhat useful for this application. Analysis of variance 
between the question responses and the DoD employer categories 
also showed that a relationship exists. The majority of the 
participants (87), represented by the U.S. Army, DCMC, and 
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DCAA, responded that the methods were somewhat useful where 
the remaining respondents (31) answered that the methods were 
very useful to the process. 



Table 4 

Analysis of Variance For Benchmarking Process Costs 



Job Classification 


DF 






F 


P 


Between Groups 


8 


11.91 


1.49 


2.09 


.04 


Within Groups 


120 


85.30 


.71 






DoD Component 


DF 






F 


P 


Between Groups 


6 


10.55 


1.76 


2.95 


.01 


Within Groups 


111 


66.07 


.60 







Source: Developed by researcher. 

Regression analysis for this question displayed 
relationships between the respondent's level of mathematics 
education and their answers. The mathematical nature of 
parametric cost estimating methods led to the belief that as 
one's mathematics education increases so does familiarity with 
the concepts employed in them. This in turn should lead to 
greater confidence in the proper application of the methods by 
these same individuals . There also was a relationship between 
General Service/Senior Executive Service (GS/SES) level or 
military rank for Federal Government employees and their 
responses. In both of these regression analyses, the groups 
observed agreed with the main body of respondents in the 
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degree of utility for parametric cost estimating methods in 
benchmarking process costs. These two groups displayed a 
specific trend beyond that initial observation, however. 

The regression analysis for the mathematics education 
relationship yielded a p value of .02 for the six subgroups 
reviewed. This value indicates a relationship does exist 
between the data points, which can be described as weak in the 
following equation: 

Y = -.337(X) + 38.3 = .033 

(.14) (.40) 

(The values in parentheses represent standard errors of the 
above coefficients.) 

In this analysis, as an individual's mathematics education 
increases the numerical value of their responses decreases. 
A lower value indicates greater utility because of the inverse 
construction of the response scale used where a one indicated 
the greatest utility and a five the least utility. The small 
value of the coefficient of determination, , shows that 
little of the variation in the dependent variable, responses, 
is due to the independent variable, mathematics education. 

Similar results were observed in the analysis comparing 
the responses with the GS/SES level or military rank. The 
regression equation for this analysis 

Y = -.445(X) + 8.68 R^ = .134 

(.21) (2.98) 
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shows that as seniority increases, so does the respondent's 
opinion of the utility of parametric cost estimating for 
benchmarking process costs. This relationship can also be 
described as weak. 

Although there were a large number of people, who did not 
choose to offer an opinion, most of those that did saw some 
promise in using parametric cost estimating methods for 
benchmarking process costs. Groups indicating a high degree 
of utility for them also are typically in an oversight, upper 
management position or those working to directly support 
others in those positions. These individuals are more likely 
to view a problem in a systems or process approach and may be 
more experienced with using parametric cost estimating methods 
in the area of benchmarking process costs than the group as a 
whole . 

This area of cost analysis is one that may take on more 
prominence in the future with parametric cost estimating 
methods playing a significant role. As DoD reliance on 
military specifications and standards is reduced, the cost 
estimating techniques built on those standards will become 
less reliable. Greater dependence on performance 
specifications may require the DoD acquisition community to 
analyze commercial processes that have had little or no prior 
application in DoD acquisition processes. The DoD Single 
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Process Initiative (SPI) may provide opportunities to use 
parametric cost estimating methods to benchmark process costs. 
With supervisory personnel experienced in and amenable to 
their use, parametric cost estimating methods should see 
additional future applications in benchmarking process costs. 

2 . Validating Bottoms Up Estimates 

Traditionally, using parametric cost estimating methods 
in this area involves comparisons between the parametric cost 
estimate and the project's bottoms up estimate. With a long 
history of use in these applications, the aggregate data 
showed a strong trend toward acceptance of the idea that 
parametric cost estimating methods were somewhat useful to 
validate bottoms up estimates. Only 20 responses were coded 
for no use or a hindrance to validating bottoms up estimates 
while there were 140 responses indicating opinions that the 
methods were at least somewhat useful in these situations. 
Group breakdown data of interest are presented in Table 5 
while Table 6 contains analysis of variance data. 

Examination of the analysis of variance data showed those 
identifying themselves as engineers and program managers 
believed that parametric cost estimating methods were more 
useful than did the other job classification groups. There is 
a significant difference between the upper level management's 
opinion of their utility and that of their supporting 
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organizations. Support groups are represented by the program 
staffer and engineer group data in Table 5. 



Table 5 

Validating Bottoms Up Estimates Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Program Staff 


3.00 


3.00 


3.00 


3.00 


.00 


3.00 


3.00 


Corporate 

Policy 


1.00 


2.00 


1.40 


1.00 


.55 


1.00 


2.00 


Engineers 


1.00 


4.00 


2.88 


3.00 


1.1 


2.00 


4.00 


Program 

Managers 


1.00 


3.00 


1.50 


1.00 


.84 


1.00 


2.25 



Source: Developed by researcher. 



Table 6 

Analysis of Variance For Validating Bottoms Up Estimates 



Job Classification 


DF 






F 


P 


Between Groups 


9 


15.37 


1.71 


2.45 


.01 


Within Groups 


144 


100.53 


.70 







Source: Developed by researcher. 

The program managers and those who set corporate policy 
felt that the methods were vital or very useful in this area 
while the engineers and program staffers thought that the 
methods were only somewhat useful in this functional area. 
This divergence of opinion may be a result of program staffers 
and engineers who had closer involvement with the techniques 
and methods than the program managers and those who set 
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corporate policy. The weak negative relationship exhibited in 
the regression analysis 

Y = -.119(X) + 7.11 r2 = .033 
(.05) (1.90) 

supports the contention that the upper level supervisors have 
a higher opinion of the method^ s utility than do the support 
organizations. The small value indicates a weak 
relationship, however. 

Parametric cost estimating methods will continue to be 
used to validate bottoms up estimates. This function is one 
of the traditional uses for parametric cost estimating for 
which there is still a need. A large part of the DoD 
acquisition community is familiar with their application in 
validating estimates derived from other sources. Certain 
situations lend themselves to the development of bottoms up 
estimates, and the subsequent need to have them validated 
against some standard will always remain. These factors 
should continue to encourage the use of parametric cost 
estimating methods in validating bottoms up estimates. 

3. Program Estimates 

Developing program estimates is also an area of 
application for parametric cost estimating. Consideration of 
the total costs for a program or system has always been a 
concern and is becoming more important in acquisition 
planning. In the aggregate analysis, the majority of the 
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participants felt that the methods were useful to some degree 
in developing program costs. Only seven respondents answered 
that there was no use for parametric cost estimating in 
program estimating and no one responded that they were a 
hindrance. The average response value for the entire database 
was 2.25 with a standard deviation of .82. Demographic group 
statistics of interest are shown in Table 1 , and Table 8 
contains analysis of variance information. 

In the examination of the analysis of variance data for 
job classifications, program managers, engineers, and contract 
negotiators had somewhat lower group response averages than 
did the remaining job classification groups represented in 
Table 7. Analysis of variance of the DoD component responses 
also showed some interesting trends. The DCAA group's 
response average was higher than that of every other group 
while the industry group had the lowest response averages. 
The DCAA group opinions are noteworthy as they represent a 
fairly homogenous response from a large subset, 33 responses 
to this question, of all of the responses. DCAA has a 
significant part in the DoD acquisition process. They have 
published guidance on the auditing procedures for parametric 
cost estimating methods. 

The NASA responses may be an outgrowth of the nature of 
the work NASA performs. NASA has a great deal of experience 
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with using parametric cost estimates in numerous developmental 
projects and may be more experienced in their use than the DoD 
organizations. The NASA respondents also have one of the 
highest average levels of mathematics education of the groups 
surveyed. The DCAA responses may reflect distance from the 
program estimating functions and a different definition of 
what constitutes program estimating than that used in the 
other groups. Also of note is that the DCAA group had one of 
the lowest mathematics education levels of the groups 
surveyed. These factors may combine to produce the divergence 
of opinion in the utility of parametric cost estimating 
methods between these two groups . 



Table 7 

Program Estimates Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Program 

Managers 


1.00 


3.00 


1.29 


1.00 


.76 


1.00 


1.00 


Industry 


1.00 


3.00 


1.47 


1.00 


.72 


1.00 


2.00 


Contract 

Negotiators 


1.00 


2.00 


1.50 


1.50 


.58 


1.00 


2.00 


NASA 


1.00 


4.00 


1.57 


1.00 


1.13 


1.00 


2.00 


Engineers 


1.00 


3.00 


1.78 


2.00 


.83 


1.00 


2.50 


DCAA 


1.00 


4.00 


2.76 


3.00 


.71 


2.00 


3.00 


Program 

Staff 


3.00 


3.00 


3.00 


3.00 


.00 


3.00 


3.00 



Source: Developed by researcher. 
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Table 8 

Analysis of Variance For Program Estimates 



Job Classification 


DF 






F 


P 


Between Groups 


8 


19.17 


2.40 


4.43 


.00 


Within Groups 


141 


76.33 


.54 






DoD Component 


DF 






F 


P 


Between Groups 


6 


14.11 


2.35 


4.41 


.00 


Within Groups 


132 


70.32 


.53 







Source: Developed by researcher. 

The data and statistics for this question appear to have 
contradictory indications. The program manager and program 
staff groups had different opinions of the utility of 
parametric cost estimating methods; the program managers felt 
they were vital or very useful while the program staff felt 
that they were only somewhat useful. The experience and 
mathematics education response values for these two groups may 
explain this divergence. The program managers have a 
significantly higher mathematics education level on average 
than do the program staff but a lower experience average. 

The relationships between the two regression analyses 
seems to be contradictory. Two of the groups with the most 
experience have widely divergent opinions about the utility of 
the methods in program estimating. One would expect that as 
experience increases so would someone's familiarity with 
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parametric cost estimating methods, their uses and 
limitations. The experience regression equation 

Y = .261 (X) - 5.13 R2 = .053 

(.08) (2.39) 

indicates a weak positive relationship exists. More 
experience leads to a belief that the methods are less useful 
in program estimating. The mathematics education regression 
analysis 

Y = -.118 (X) - 7.03 R2 = .033 

(.05) (1.75) 

shows a weak negative relationship. The DCAA responses 
exemplify these two equations. Their view of the utility of 
the methods in this functional area is low when compared to 
the other groups, and they are one of the most experienced 
groups, but they also have one of lower mathematics education 
averages of the groups. 

The DCAA group had lower opinions of the utility of 
parametric cost estimating methods in this application than 
did the other groups. This may represent a more detailed 
working knowledge of the uses of the methods and their 
limitations than the other groups had. It also may be a 
result of a different view of what is included in the 
definition of a program estimate. These groups may have used 
a more restrictive definition of a program estimate than did 
the remaining groups. In restricting the definition to 
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specific fiinctional estimates produced at various times during 
a program cycle, they may have been more skeptical about using 
parametric cost estimating methods than the program managers 
who may have taken a more expansive view of the definition. 
The majority of the groups in Table 7 show a trend towards 
acceptance of the use of these methods for producing program 
estimates . 

4 . Budgeting 

Budgeting is another area in which parametric cost 
estimates have a long history of use. They have been produced 
for rough order of magnitude budgeting as well as for more 
specific estimates in the budgeting process. - The overall 
database information had all but nine participants indicating 
their belief that parametric cost estimating had some use in 
budgeting decisions. The majority of the respondents chose 
the descriptor very useful as their response to the question. 
The average response value was 2.2 with a standard deviation 
of .85 for the responses. Table 9 displays the group 
breakdown data applicable to this question with Table 10 
presenting analysis of variance data. 

The subgroups identified as cost or price analysts and 
contracting officers were of note in the analysis of variance. 
Both of these groups represent large sections of the database, 
45 and 36 responses to this question respectively, with 
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relatively small standard deviations for both groups. The 
U.S. Army and DCAA subgroups were also identified in the 
analysis of variance as being of note. These two groups also 
represent major parts of the database, and when compared with 
the other DoD component groups, they show relatively tight 
data groupings with response average values larger than those 
of the other DoD component groups. These four groups all 
responded that the use of parametric cost estimating methods 
were of some use in the budgeting process. 

Three other groups, program managers, contract 
negotiators, and those that set corporate policy, responded 
that the methods are vital or very useful to the budgeting 
process. The responses from the program managers and industry 
are of special interest. Of the groups who would be expected 
to have considerable experience in budgeting, they have the 
highest opinion of the utility of the methods in budgeting. 
The data from the DCAA and program manager groups are 
supported by the mathematics education regression analysis. 
The equation 

Y = -.137(X) + 7.33 R2 = .051 

(.04) (1.66) 

indicates that there is a weak negative relationship 
demonstrated in the DCAA high average response to this 
question and relatively low mathematics education average when 
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compared with other groups. The program managers have 
opposite characteristics with one of the highest mathematics 
education levels and low average response to this question. 



Table 9 

Budgeting Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


U . S . Army 


1.00 


4.00 


2.51 


3.00 


.73 


2.00 


3.00 


DCAA 


1.00 


4.00 


2.63 


3.00 


.77 


2.00 


3.00 


Cost or 

Price 

Analysts 


1.00 


4.00 


2.23 


2.00 


.70 


2.00 


3.00 


Contracting 

Officers 


1.00 


4.00 


2.28 


2.00 


.78 


2.00 


3.00 


Program 

Managers 


1.00 


3.00 


1.57 


1.00 


.79 


1.-00 


2.00 


Contract 

Negotiators 


1.00 


2.00 


1.25 


1.00 


.50 


1.00 


1.75 


Corporate 

Policy 


1.00 


2.00 


1.60 


2.00 


.55 


1.00 


2.00 



Source: Developed by researcher. 



Reviewing the results of the analysis leads one to 
believe that parametric cost estimating applications in 
budgeting should continue their long history of use. As the 
data show, there are a large number of groups in the DoD 
acquisition community who feel strongly that parametric cost 
estimating methods are useful techniques for budgeting. The 
increased prominence that they are given through the 
Parametric Cost Estimating Initiative will widen the knowledge 
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about their uses in the community. The increasing accuracy 
and speed at which these type of estimates can be generated 
make them a formidable tool for contingency analyses in an 
environment of funding uncertainty. From their already wide 
acceptance base, this should lead to more applications in 
budgeting in the future. 



Table 10 

Analysis of Variance For Budgeting Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


20.97 


2.33 


3.97 


.00 


Within Groups 


145 


85.22 


.59 






DoD Component 


DF 






F 


P 


Between Groups 


6 


10.48 


1.75 


2. 92 


.01 


Within Groups 


131 


78.34 


. 60 







Source: Developed by researcher. 

5. Independent Cost Estimate (ICE) Generation 

Generating independent cost estimates is an important 
part of any acquisition planning process. They may be used in 
any number of program functions to include program estimates 
or budgeting. Parametric cost estimating is one of the tools 
that can be applied to generate these estimates. Responses to 
this segment were centered between very useful and some use. 
No responses reflected the opinion that the methods would be 
a hindrance to generating ICE generation. There were a large 
number of no responses to this question, however. The average 
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response value was 2.34 and the median value was 2, or very 
useful. In the analysis of variance the engineers believed 
that the use of parametric cost estimating had the most 
application to ICE generation of the groups. They indicated 
that the methods were very useful while the remaining job 
classification groups felt that the methods were only somewhat 
useful for ICE generation. Tables 11 and 12 contain the 
group specific data and analysis of variance for this segment 
of question one. 



Table 11 
ICE Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control Group 


1.00 


3.00 


2.00 


2.00 


.94 


1.00 


3.00 


USAF 


1.00 


3.00 


2.07 


2.00 


.73 


1.75 


3.00 


Engineers 


1.00 


3.00 


1.46 


1.00 


.69 


1.00 


2.00 



Source : 



Developed by researc 



her. 



Table 12 

Analysis of Variance For ICE Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


16.83 


1.87 


2.29 


.02 


Within Groups 


104 


84.79 


.82 







Source: Developed by researcher. 

The responses of the engineer job classification group 
indicated the highest level of regard for the utility of 
parametric cost estimating for producing ICEs. This group is 



84 



representative of those in the DoD acquisition community 
tasked with ICE generation and who are familiar with the 
process. They also have one of the highest average 
mathematics education levels of any group in the database. 
These factors indicate a viable application for using 
parametric cost estimating methods in ICE generation exists, 
and there is a base of expertise in their use in this area. 

The high regard that the control and USAF groups had for 
the use of parametric cost estimating methods for ICE 
generation is also indicative of their application potential. 
The control group has been studying the use of the methods and 
they have responded that the techniques are very useful in 
this area. Regression analysis revealed the following 
relationship between answers for this question and the 
respondents' mathematics education levels. 

Y = -.169 (X) + 8.74 R2 = .062 
(.06) (2.14) 

The weak negative relationship between the answers and the 
mathematics education responses supports the observation that 
these groups would strongly support the use of parametric cost 
estimating methods for ICE generation. 

The application of parametric cost estimating methods to 
producing ICEs will continue and become more prevalent. DoD 
Regulation 5000. 2-R states, "As defense resources decline, the 
margin available for adjustments narrows, making realistic 
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cost projections all the more critical." [Ref 44] Parametric 
cost estimating has and will continue to fulfill a major role 
in producing ICEs to meet this need. The groups cited in 
Table 11 are important members of the DoD acquisition 
community who favorably view the application of parametric 
cost estimating methods to ICE generation. With the emphasis 
from the control group on the potential uses of the methods, 
the engineers representing the users of the methods in this 
area, and the auditor group noting that the methods have some 
application in the production of ICEs, they should certainly 
find wider use in future procurement actions and planning. 

6. Software Contracts 

There are a host of databases and computer programs that 
were developed for the purpose of estimating costs and 
schedules for software development. Examples such as COCOMO 
and PRICE-S were cited earlier in the text. The cumulative 
data showed that the largest part of the respondents indicated 
that parametric cost estimating methods had some use in 
software contracts. Few of the participants (27) thought that 
they had no use. The average response value to this question 
was 2.54 and a median value of 3 was observed from the 
aggregate data. 

In reviewing the group data, the USAF was observed to 
have a somewhat higher regard for the application of 
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parametric cost estimating methods to software contracting 
than the main body of the participants. USAF group data, an 
average response value of 2.07 with a .616 standard deviation 
and median value of 2, indicate a view that the methods are 
very useful in this area. The USAF has been greatly involved 
in the use of parametric cost estimates for estimating 
software development costs. The USAF Cost Analysis Agency has 
produced a revised COCOMO model for this express purpose. 
This versatile model can be used to produce a variety of 
outputs throughout the life cycle of a software product. 

Parametric cost estimating has a strong base of 
application as a tool in the area of software cost estimating. 
Estimating the costs of and schedule for software development 
has been a historically difficult area. The increasing 
importance of software in DoD systems and equipment could make 
this a growth area for the future. Parametric cost estimating 
methods should become more useful for software cost estimate 
generation as databases become more interconnected. 

7 . Repair Parts Contracts 

Aggregate data analysis presented a view that parametric 
cost estimating methods had some utility in producing cost 
estimates for repair parts contracts. The average response 
value was 2.85, a median value of 3 and a normal distribution 
of the data around that value was observed. Table 13 shows 
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the group data for contracting officers and those that set 
Government policy. The groups shown are noted for their 
concurrence within their groups demonstrated by the small 
standard deviations identified for each group. Both groups 
can also potentially play a major part in .the use of 
parametric cost estimating in repair parts contracting. Those 
that set Government policy (through regulation or directive) 
and contracting officers (through applying the methods in 
specific procurement actions) have the potential to directly 
encourage or discourage the use of parametric cost estimating 
methods in this area. That these groups both indicate that 
they see limited use for the methods in repair parts 
contracting would seem to limit their uses here. 

Repair parts contracting is not normally an area in which 
the methods of parametric cost estimating are applied. 
Generally the costs or prices are fairly well established 
through historical prices. The need for estimating repair 
parts cost or prices typically occurs later in the life cycle 
than the stages in which parametric cost estimating methods 
are used. Parametric cost estimating methods could be used in 
an overall estimate of repair parts costs for a system or 
piece of equipment much more readily than for an individual 
procurement action. The data reflect a preference for other 
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cost or price estimating methods to estimate repair parts 
costs over the use of parametric cost estimating methods. 



Table 13 

Repair Parts Contracting Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Contracting 

Officers 


2.00 


4.00 


2.85 


3.00 


.57 


2.50 


3.00 


Government 

Policy 


2.00 


4.00 


3.05 


3.00 


.67 


3.00 


3.50 



Source: Developed by researc 



her . 



8 . R&D Contracts 

R&D contracting is an area in which parametric cost 
estimating can be applied in a number of areas. The types of 
items and services procured in R&D contracting are 
developmental in nature, and estimating costs or prices can be 
difficult. The database analysis for the entire response 
group yielded an almost equal distribution between the very 
useful, somewhat useful and no use descriptors, 41, 49, and 40 
responses respectively. An average response value of 2.83 
indicates a somewhat useful viewpoint throughout the DoD 
acquisition community regarding the utility of parametric cost 
estimating methods to R&D contracting. The NASA participants 
had the highest regard for their applicability in the analysis 
of variance. Job classification and DoD component data for 
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the analysis of variance is displayed in Table 15. DCAA 
responses indicated that in their view the methods were of 
some or no use to R&D contracting. 



Table 14 

R&D Contracting Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


DCAA 


2.00 


5.00 


3.53 


4.00 


.80 


3.00 


4.00 


USAF 


1.00 


4.00 


2.21 


2.00 


.89 


2.00 


2.25 


NASA 


1 . 00 


3.00 


1.67 


1.50 


.82 


1.00 


2.25 


Technical 

Support 


3.00 


4.00 


4.00 


4.00 


1.00 


3.00 


4.00 



Source: Developed by researcher. 



The NASA and USAF data indicate that there are some in 
the DoD acquisition community that feel that parametric cost 
estimating methods can be used in R&D contracting. It is also 
of interest that the technical support group participants were 
the group that had the highest average response rate of all 
the groups. The response rate shown in Table 14 indicates 
that this small group believes that the methods have no use in 
R&D contracting. 

The mathematics education regression analysis supports 
the observations regarding the groups in Table 14. The 
regression equation 

y = -.200(X) + 10.3 R2 = .078 
(.05) (2.01) 
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indicates that there is a weak negative relationship between 
mathematics education and the responses for this question. 
The NASA participants had a higher average mathematics 
education level than did the other groups and they also felt 
that the methods had more use in R&D contracting .than did the 
other respondents . 



Table 15 

Analysis of Variance For R&D Contracting Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


31.44 


3.49 


3.62 


.00 


Within Groups 


138 


133.29 


.97 






DoD Component 


DF 






F 


P 


Between Groups 


6 


28.32 


4.72 


5.42 


.00 


Within Groups 


127 


110.61 


.87 







Source: Developed by researcher. 

The application of parametric cost estimating methods 
should continue to be used in this area where the groups 
mentioned have found them useful. Increasing budgetary 
pressures have restricted spending on R&D throughout DoD and 
it is likely that R&D funding will be restricted in the 
future. A lack of useful databases in the R&D field may be 
contributing to the view of almost one third of the 
respondents that parametric cost estimating methods are of no 
use. While parametric cost estimating cannot be used for 
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every instance, the methods do have the potential to produce 
sound estimates for R&D applications. 

9 . Service Contracts 

Acquisition of services forms a significant part of the 
Federal Government's procurement activity. Parametric cost 
estimating methods have not been widely applied to this area 
of acquisition in the past. The majority of the responses to 
this segment were split between some use and no use with 57 
responses for each choice. The average response value from 
the database was 3.09 and the median response value was 3. 
These indicate that the participants felt there was only some 
use for the methods in service contracting. The results of 
the analysis are shown below in Table 16. 

Service contracting usually does not have the 
characteristics that make it amenable to using parametric cost 
estimating techniques. The procurements in service 
contracting are not often quantifiable in terms of parameters 
that are easily measured and then related to outputs in a 
database. There are few existing databases and software 
products that are used in this area. The control group 
average response rate supports this view in their opinion that 
parametric cost estimating methods do not have any use in the 
area of service contracting. The rather large standard 
deviation indicates a lack of consensus within this group, 
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however, but at no point does anyone in the group advocate 
that the methods have any more than some use in service 
contracting. 



Table 16 

Service Contracting Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Set Corporate 
Policy 


3.00 


4 . 00 


3.25 


3.00 


.50 


3.00 


3.75 


Control Group 


3.00 


5.00 


4.17 


4.00 


.98 


3.00 


5.00 



Source: Developed by researcher. 



The group of respondents that set corporate policy 
differs in their opinion but only slightly. Their answers 
lead one to believe that they feel that the methods have some 
use in this area of contracting. Unlike the control group, 
those that set corporate policy have a closer consensus 
regarding the methods' application in service contracting 
evidenced by the small standard deviation in a group twice the 
size of the control group. 

Defining services to be acquired by performance standards 
may allow greater use of parametric cost estimating methods in 
service contracting. Performance specifications are the 
preferred method of describing supplies or services required 
by the Federal Government. With this emphasis and increased 
awareness of parametric cost estimating methods, the 
techniques may see more use in this area of Federal Government 
procurement . 
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10. Hardware Contracts 



Although hardware can be defined in terms of components, 
assemblies or subassemblies of an electrical, mechanical, or 
electronic piece of equipment, the definition used here is 
more limited. Hardware contracting for this study refers to 
contracting for computer hardware, equipment, or peripherals. 
The rapid pace of development in the computing field leads to 
the use of parametric cost estimating in this area. The data 
and analysis apply equally as well to a broad definition of 
hardware as they do to the narrower definition used in this 
study. 

Database analysis revealed that the majority of the 
respondents answered that the methods were very useful or of 
some use in hardware contracting, 54 and 59 responses 
respectively. Some use was the median response and an average 
response value was 2.55. Contracting officers were the only 
group that showed a consensus, a standard deviation of .62 was 
observed, in their responses to this question. In their 
position, they are also one of the groups that could best 
apply the methods to contracting for hardware. Their 
responses with a mean of 2.65 and median value of 3 indicate 
a view that the methods are of some use in this area. 

While the methods may have application in producing 
estimates to be used in the area of hardware contracting, the 
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drive to use Commercial-of f-the-Shelf Technology (COTS) and 
Non-developmental Items (NDI) may restrict their uses by the 
Federal Government in this area in the future. COTS items and 
NDI are by definition beyond the stage in which parametric 
cost estimating methods are typically applied in, the product 
life cycle, as shown in Figure 1. If the item is truly a COTS 
item or NDI there should be ample pricing data available from 
the commercial environment and less need to apply the 
techniques to this area of Federal Government acquisition. 
This type of acquisition environment may, however, hold some 
promise for the use of parametric cost estimating methods. 
The methods may be used to estimate the costs of integrating 
the various COTS items and NDI that may be purchased to meet 
an agency's needs rather than for the equipments themselves. 

11. Construction Contracts 

Parametric cost estimating methods have been successfully 
applied to many construction projects. There are plentiful 
data and output parameters in construction that lend 
themselves to using parametric cost estimating methods. The 
CPRG uses a construction example to illustrate the development 
of concepts and CERs in parametric cost estimating. Some use 
was the choice of 54 of the respondents to the survey. There 
were a significant n\jmber of participants (79) who elected to 
not respond to this segment of question one. An average 
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response value for the participants of 2.71 was observed and 
the median response was the some use descriptor. The group 
response data are presented below in Table 17. 



Table 17 

Construction Contracting Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Contracting 

Officers 


2.00 


4.00 


2.79 


3.00 


.78 


2.00 


3.00 


Negotiators 


1.00 


2.00 


1.75 


2.00 


.50 


1.25 


2.00 


Government 

Policy 


2.00 


4.00 


2.86 


3.00 


.72 


2.00 


3.00 



Source; Developed by researcher. 



The contract negotiator response is of particular 
interest in Table 17. This group indicated that they believed 
that there was greater utility for parametric cost estimating 
methods in construction contracting than did the other 
participants and groups. The contract negotiators indicated 
the methods were vital or very useful in the area of 
construction contracting while others felt that they were only 
of some use in the process. 

The contract negotiators' consensus on this question is 
also noteworthy when compared with the responses from the 
contracting officers. With similar training and experiences, 
one would expect similar responses from both groups. The 
contracting officers, however, felt that the methods were only 
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of some use in construction contracting. These individuals 
and those that set Government policy can be most influential 
in applying the methods in this area, and both do not see the 
methods contributing significantly in this area. 

Parametric cost estimating methods should continue to be 
employed in estimating for construction contracting. They 
have an established basis for use outside the DoD and a large 
part of the DoD acquisition community involved in construction 
contracting is aware of how they can be best applied to this 
area. Base consolidations and relocations could offer 
additional opportunities for using parametric cost estimating 
methods in construction projects. 

12. Commodity Contracts 

Contracting for commodities, such as fuels, clothing, or 
food items, is a significant and vital part of the DoD 
acquisition workload. Commodities may defined as commercial 
items sold in substantial quantities to the general public 
exemplified by the group and class categories in the GSA 
catalogs. Aggregate group analysis showed that the majority 
of the responses to this question were split between some use 
and no use in this area, 51 and 41 responses respectively. 
The analysis also showed a median response value of 3 and 
average response value of 3.23 with a standard deviation of 
.78. The small standard deviation indicates that a consensus 
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on this q[uestion exists. No group in the analysis deviated 
from this opinion. 

Pricing for commodities is normally accomplished using 
market pricing mechanisms. As a result, there are few 
opportunities for the use of parametric cost estimates. The 
aggregate group data presented support the idea that the 
methods have limited application in commodity contracting. 
This situation is unlikely to change significantly because 
market prices are one of the preferred pricing methods for 
commodity items . 

C. SUMMARY 

Question one was broken into 12 segments to .take a broad 
view of some of the major areas in the acquisition process to 
which parametric cost estimating may be applied. The results 
of the analysis conducted across these areas showed the 
opinions of the DoD acquisition community regarding the 
applicability of the methods. Overall, the methods were seen 
to be most applicable to the program estimating, budgeting and 
ICE generation and least applicable to commodity contracting. 
In all of the other areas the respondents indicated there was 
some use for the techniques . 
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VI. SPECIFIC APPLICATIONS 



A. PURPOSE 

Questions two through four ask the participants to rank 
the utility of parametric cost estimating methods across a 
series of specific applications in program management and 
contract situations. Question two addresses system life cycle 
concerns as defined in DoD Regulation 5000. 2-R. Figure 1 
presents a typical pattern of applying the methods throughout 
the four life cycle phases. Question three presents data on 
the uses of parametric cost estimating methods in specific 
contract type situations. Finally, question four addresses 
applications in pre-award and post-award contract actions. 

B. PROGRAM LIFE CYCLES PHASES 

1 . Concept Exploration 

In the first phase of a system' s life cycle, concept 
exploration (CE) , a series of short term, independent studies 
are conducted. These studies attempt to evaluate various 
concepts that are proposed to meet a requirement. Parametric 
cost estimating, as shown in Figure 1, is not the primary cost 
estimating method in this phase. The aggregate responses were 
essentially evenly spread across the three utility descriptors 
and no responses. Vital, very useful, some use, and no 
responses garnered 39, 48, 43, and 39 answers respectively. 
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The dispersion of responses indicates a lack of agreement on 
the answer to this question which is also identified in the 
large standard deviation of 1.09. The average response value 
of 2.46, a median value of 2 and a positively skewed 
distribution indicate that the participants felt that the 
methods were very useful in this life cycle phase. 

The job classification analysis of variance data 
highlighted the control and NASA groups. Their responses 
indicated that for this phase they felt the methods had the 
most utility of all of the groups. The auditor job 
classification group was identified as believing that the 
methods had the least degree of utility in this life cycle 
phase. These results may be an outcome of the experience that 
the control and NASA groups have with this phase of program 
management and the lesser degree of experience that the 
auditors may have with it. Tables 18 and 19 contains these 
data. 

The low response averages for the program staff, NASA, 
and control groups for this question are noteworthy. These 
groups represent high mathematics education levels and are 
familiar with the actions required in each of the phases of a 
system's life cycle. The mathematics regression analysis also 
supports this. The regression equation and data 

Y .271 (X) +12.6 r2 = .132 

(.05) (1.99) 
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show a weak negative relationship between the responses and 
the mathematics education level of the participants. These 
groups indicated that they believed that the methods were very 
useful in the concept exploration phase. The data from 
analysis of the auditor and contracting officer responses show 
that they feel that the methods are only of some use in this 
phase . 



Table 18 

Concept Exploration Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Program 

Staff 


2.00 


2.00 


2.00 


2.00 


.00 


2.00 


2.00 


Control 

Group 


1.00 


3.00 


1.45 


1.00 


. 69 


1.00 


2.00 


NASA 


1.00 


4.00 


1.50 


1.00 


1.07 


1.00 


1.75 


Auditors 


1.00 


4.00 


3.13 


3.00 


.91 


2.00 


4.00 



Source: Developed by researcher. 



Table 19 

Analysis of Variance For Concept Exploration Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


28.99 


3.22 


2.97 


.00 


Within Groups 


149 


161.35 


1.08 






DoD Component 


DF 






F 


P 


Between Groups 


6 


26.25 


4.37 


4.17 


. 00 


Within Groups 


134 


140.73 


1.05 







Source: Developed by researcher. 
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The CE phase activity is centered on reviewing various 
solutions to a problem vice analyzing a specific solution. It 
is not until a general course of action is decided on that 
system measures of effectiveness or performance can be 
developed. Without some general definition of how a solution 
to the problem is to be described, a database cannot be 
selected for analysis nor are there any parameters to 
evaluate. Once this has been accomplished, parametric cost 
estimating methods can be successfully used in this area of 
program management. In this phase, they may have more 
application as rough order of magnitude planning tools in 
budgeting or for conducting cost and performance tradeoff 
analyses early in a program than for specific cost or schedule 
estimates . 

2. Program Definition & Risk Reduction 

Activity in the PD&RR phase centers on the evaluation of 
the problem solution identified at the Milestone I decision 
point. Defined in DoD Regulation 5000. 2-R, these activities 
may include prototyping, testing, and early operational 
assessment of critical systems, subsystems, and components. 
[Ref 43:p.4] Actions, such as these, are taken to identify 
and reduce risk in the early stages of a program. Responses 
to this segment of question two were concentrated on very 
useful, 60 responses, and some use, 58 responses. An average 
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response value of 2.43 and median answer of very useful was 
observed. 

Analysis of variance results highlighted the corporate 
policy, NASA, program staff, and control groups as 
particularly interesting. These groups believe that the 
methods are vital to this phase of a system's life cycle. In 
fact, the program staff members were unanimous in their 
opinion that the methods were very useful in the PD&RR phase. 
Data analysis results are displayed in Tables 20 and 21. The 
view expressed by the groups that the methods have a great 
deal of utility in this phase is noteworthy. Most of the 
groups work in this area of program management or have 
experience in performing the functions in this phase and 
represent a cross section of upper level management in the DoD 
acquisition community. 



Table 20 

PD&RR Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Program 

Staff 


2.00 


2.00 


2.00 


2.00 


.00 


2.00 


2.00 


Control 

Group 


1.00 


2.00 


1.55 


2.00 


.52 


1.00 


2.00 


Corporate 

Policy 


1.00 


2 . 00 


1.20 


1.00 


.45 


1.00 


1.50 


NASA 


1 . 00 


4 . 00 


1.57 


1.00 


1.13 


1.00 


2.00 



Source: Developed by researcher. 
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Figure 1 presents the view that parametric cost 
estimating is most useful in this stage of a system's life 
cycle. This is demonstrated by the response averages and 
standard deviations of the groups shown in Table 20. The 
following regression equations support this observation. 

Mathematics Education 

Y .201 (X) + 9.95 r2 = .099 

(.05) (1.70) 

GS/SES Level/Rank 

Y = -.343(X) + 7.09 R^ = .094 

(.16) (2.24) 

Although these relationships can be described as weak, they do 
indicate that as one' s mathematics education and rank increase 
the belief in the utility of parametric cost estimating 
methods to PD&RR becomes stronger. 



Table 21 

Analysis of Variance For PD&RR Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


15.58 


1.73 


2.12 


.03 


Within Groups 


153 


124.73 


.82 






DoD Component 


DF 






F 


P 


Between Groups 


6 


13.37 


2.23 


2.93 


.01 


Within Groups 


138 


140.86 


.76 







Source: Developed by researcher. 

Applications for parametric cost estimating in the PD&RR 
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protessionais siiouid ensure tiiat tiiey are used where they will 
provide the most benefit. 

3. Engineering & Manufacturing Development 

In the Engineering & Manufacturing Development phase 
(E&MD) , one of the functions is to define and- perfect the 
manufacturing processes to be used to produce the equipment 
selected at the Milestone II decision point. These processes 
provide opportunities for the use of parametric cost 

"nn v -i -r% T? rlWTn TT-i n v-o 1 o V> /-nt.to 
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estimates also. This phase is the transition point between 
the defined solution and production of equipment representing 
that solution. 

Participants responded to this question with 62 responses 
in the very useful and 76 responses in the some use descriptor 
categories. There was an average response value of 2.45 and 
the median response was for some use in the E&MD phase. 
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and those who set corporate policy were interesting. 
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the methods here than do the cost estimators. 



The cost 



estimators believed, as did the DCAA group, that the methods 
have limited application in the E&MD phase while those who set 
corporate policy believe that it is vital to the process to 
use them in the E&MD phase. This divergence of opinion 
between these groups is noteworthy in that all of these groups 
have a major role in program management at this phase. Tables 
22 and 23 contain the group analysis and analysis of varia.nce 

— » -f~ — I Tr ^4’Th -v~ r' 
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laDre zz 
E&MD Group Data 



, 


1 

Min 


1 1 
Max 


1 1 
II 

r' 


1 1 

Median 


1 1 

a 


01 


03 


Cost 

TT r~> "J— -i -rr> — V -I- •v~ <-i 

XjO 1..,X-XUCX O 


1 . 00 


3 . 00 


2 . 96 


3 . 00 


. 66 


2 . 00 


3 . 00 


Pr^'T'nr^-r » +• a 

V.- 

1 "i T 7 

X JL JL\^ y 


1 - 00 


2 . 00 


1 - 40 


1 . 00 


.55 


1 . 00 


2.00 


DCAA 


1.00 


4.00 


2.79 


3.00 


.70 


2.00 


3.00 


U.S. Army 

1 


1.00 


4.00 


2.61 


3.00 


.70 


2.00 


3.00 



Sourc0 1 D0V0lop0d r0S0arch0r . 
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applications or tne metnoas in tne same runctionai areas oy 
the commercial and Federal Government sectors of the 
acquisition community. The Federal Government interest in the 
application of business practices in the operation of 
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used with greater frequency in this phase of prograitis of all 



types , 



Table 23 



7i*r%o1 T7C*T 

XAJ.XCA.-1-jf -i- 



•p 

W A. 



V t_X J 1- CAXX \_ 



DoD Component 


DF 






F 


P 


Between Groups 


6 


8.52 


1.42 


2.51 


.03 


Within Groups 


139 


78.73 


.57 







Source: Developed by researcher. 



The existing experience ivith their uses may be expanded 
by the eiuphasis on cost “benefit tradeoffs that are now 
encouraged by DoD upper management levels. Evaluating new 
processes or processes not previously' used in production of 
DoD systems also provide additional avenues for using the 
methods more widely during the E&MD phase. Increasing 
accuracy of the parametrically derived estimates and 
increasing costs of producing bottoms up estimates should also 
enhance their uses here. 
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Support 

The final phase described in DoD Regulation 5000. 2-R is 
the Production, Fielding/Deployment & Operational Support 
phase (PF/D&OS) = In this phase, production of the eguipiaent. 
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proaucc stares. Appxicatxon or parametrxc cost estimates in 
this phase have been restricted. One such use may be in 
verifying that the program is tracking with earlier cost 
estimates by comparing them with actual production costs. 
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group data for this question. The regression analysis 

Y = .206(X) - 3.20 R2 = .024 

(.09) (2.59) 

shows a positive relationship between the responses and 
the experience level of the participants. The relationship 
between experience and the data is weak although it does 
support the idea that the experienced groups in the 
acquisition community view the methods as having limited use 
in the PF/D&OS phase. Few areas of application can 
effectively use parametric cost estimating methods in the 

PF/nxric; Tihacio ar"l-nal r'o«:'l"c: n"F Tvrr>rliir;f i on and the aDOlication 
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of the learning curve concept over the number of units 
produced are the major cost estimating methods used. 
Parametric cost estimating methods have little role to play 
and are unlikely to take on greater prominence in this area in 
the future . 

The response analysis fits the pattern of use for 
parametric cost estimating shown in Figure 1 . The 
participants as a group believe that the methods have some 
applications in the CE phase, most utility in the PD&RR and 
E&MD phases and the least utility in the PF/D&OS phase. The 
groups with the most experience using the techniques are also 
the strongest proponents of their utility in the PD&RR and 
E&MD phases. Future improvements to the techniques that 
increase their accuracy and flexibility should combine with 
the discretion given to decision makers to make sound business 
decisions to increase the uses of parametric cost estimating 
techniques in all of the program life cycle phases. 

C . CONTRACT TYPE 

Ideally, contract type is directly tied to the risk 
inherent in a particular contract situation. These situations 
are in part formed as a function of the program life cycle 
stage that the procurement represents. Figure 1 relates the 
cost risk to the program stages over time. The narrowing cost 
uncertainty band can be interpreted as declining cost risk as 
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one progresses through the life cycle of a program. The 
contract vehicles typically used in each stage also progress 
from ones that place greater risk on the buyer in earlier 
phases to ones that shift the burden of risk to the producer 
in the later stages of the program. A contract type pattern 
of use throughout a program might be described as [Ref 41] : 



• 


Phase 


0 


CPFF 




• 


Phase 


I 


CPIF 




• 


Phase 


II - 


FPIF or 


CPAF 


# 


Phase 


III - 


FPIF or 


FFP 



One of the goals of program planning is to match. the contract 
vehicle to the risks in each procurement in each phase of the 
program. Figure 8 describes a relationship between contract 
type, risk, and the utility of parametric cost estimating 
methods . 

Question three is divided into segments reflecting each 
contract type depicted in Figure 8. The response data for 
each contract type segment will be examined to determine the 
relationship between this theoretical presentation and the 
reality of the acquisition environment. Group data will be 
presented for the control, program manager, industry, and 
contracting officer groups for each contract type. These 
groups have extensive knowledge about and experience with 
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using various contract types in different procurement 
situations . 



Supplier 



Contract Risk 



Buyer 




Contract Type Continuum 

Figure 8; Contract Risk Compared With PCE Utility 
Source: Developed by researcher. 

1 . Firm Fixed-Price 

Fixed-Price contract arrangements of all types represent 
the majority of contracts written. [Ref 39:p.310] In 1994, 
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these contracts made up 75% of the contracts written by DoD. 
[Ref 45:p.81] Some use was the median descriptor response to 
the question of the utility of parametric cost estimating 
methods to Firm Fixed-Price (FFP) type contracts. This answer 
was chosen by 60 participants in the survey while .55 chose the 
very useful descriptor to describe the utility. Analysis of 
variance of the DoD components supports this opinion of the 
utility of parametric cost estimating in FFP contract 
situations. Table 24 contains group data for this question 
and Table 25 contains the analysis of variance data. 



Table 24 

FFP Contract Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control 

Group 


1.00 


5.00 


2.00 


2.00 


1.32 


1.00 


2.50 


Program 

Managers 


1.00 


3.00 


2.33 


2.00 


.82 


1.75 


3.00 


Industry 


1.00 


5.00 


2.05 


2.00 


1.13 


1.00 


3.00 


Contracting 

Officers 


1.00 


4.00 


2.58 


3.00 


.87 


2.00 


3.00 



Source: Developed by researcher. 



The U.S. Army, DCAA, U.S. Navy, and USAF groups, totaling 
105 respondents, had average response values significantly 
higher than the average. The groups identified in Table 24, 
however, make decisions about which type of contract to use in 
situations that they are presented with and thus may have more 
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influence in this area. Although there is not a consensus 
across the groups or within any one group, with the exception 
of the contracting officers, the group averages are somewhat 
lower than the average for the entire database (2.56). This 
demonstrates a view that the methods are very useful in FFP 
contracts . 



Table 25 

Analysis of Variance For FFP Contract Data 



DoD Component 


DF 






F 


P 


Between Groups 


6 


9.99 


1.66 


2.31 


.04 


Within Groups 


130 


93.77 


.72 







Source: Developed by researcher. 

2. Fixed-Price With Economic Price Adjustment 

Responses to the question of the utility of Fixed-Price 
With Economic Price Adjustment (FPE) type contracts was 
primarily that they had some use. The average response value 
was 2.71 and there was a median value of 3 to this question. 
There were also a large number (81) of no responses. The USAF 
respondents had a consensus that they were only of some use in 
these contract situations. Data from the group analysis is 
presented in Table 26. The benchmark group responses reflect 
a higher regard for the use of the methods in situations where 
one would use FPE contracts than did the entire pool of 
participants . 



113 



Table 26 

FPE Contract Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control 

Group 


I. 00 


5.00 


2.14 


2.00 


1.46 


1.00 


3.00 


Program 

Managers 


I. 00 


3.00 


2.25 


2.50 


.96 


1.25 


3.00 


Industry 


1.00 


5.00 


2.00 


2.00 


1.19 


1.00 


3.00 


Contracting 

Officers 


1.00 


4.00 


2.77 


3.00 


.81 


2.00 


3.00 


USAF 


2.00 


4.00 


2.93 


3.00 


. 62 


2.75 


3.00 



Source: Developed by researcher. 



3, Fixed-Price Incentive, Successive Target 

The aggregate group response to this question was one of 
some use for the methods in situations that a Fixed-Price 
Incentive, Successive Target (FPIS) contract arrangement was 
suitable. This question had a large number (83) of no 
responses but of those who did answer, 59 selected some use, 
and 42 selected very useful as their answers. This produced 
an average response value of 2.62 and the median of 3. Aside 
from the benchmark groups, the only group with any type of 
consensus on this question was the Government policy group. 
These participants also felt strongly that the methods were 
only somewhat useful in FPIS situations. The contracting 
officers also echoed this belief while the other benchmark 
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groups felt that the methods were very useful in these 
contract situations. Table 27 presents this information. 



Table 27 

FPIS Contract Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control Group 


1.00 


3.00 


1.83 


1.50 


.98 


1.00 


3.00 


Program 

Managers 


1.00 


3.00 


2.20 


2.00 


.84 


1.50 


3.00 


Industry 


1.00 


4.00 


1.95 


2.00 


.97 


1.00 


3.00 


Government 

Policy 


2.00 


4.00 


2.81 


3.00 


. 68 


2.00 


3.00 


Contracting 

Officers 


2.00 


4.00 


2.72 


3.00 


. 67 


2.00 


3.00 



Source: Developed by researcher. 



4. Fixed-Price With Prospective Redetermination 

Parametric cost estimating methods have essentially the 
same utility value in Fixed-Price With Prospective 
Redetermination (FPR(P)) contracts as they do in FPIS contract 
situations. The aggregate response average was 2.60 with the 
median value equal to 3 or some use. Again there were a large 
number of no responses (77) and the distribution was 
positively skewed. Analysis of variance showed that the NASA 
respondents thought that the methods were more useful than did 
the DoD component groups. The data are presented in Tables 28 
and 29. 
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Table 28 

FPR(P) Contract Group Data 





Min 


Max 




Median 


o 


Q1 


Q3 


Control Group 


1.00 


3.00 


1.83 


1.50 


.98 


1.00 


3.00 


Program 

Managers 


1.00 


3.00 


2.20 


2.00 


.84 


1.50 


3.00 


Industry 


1.00 


4.00 


1.90 


2.00 


.94 


1.00 


3.00 


Contracting 

Officers 


2.00 


4.00 


2.74 


3.00 


.67 


2.00 


3.00 


Government 

Policy 


1.00 


4.00 


2.68 


3.00 


.78 


2.00 


3.00 


NASA 


1.00 


2.00 


1.67 


2.00 


.59 


1.00 


2.00 



Source: Developed by researcher. 



Table 29 

Analysis of Variance For FPR(P) Contract Data 



DoD Component 


DF 


Sd' 




F 


P 


Between Groups 


6 


8.32 


1.39 


2.69 


.02 


Within Groups 


109 


56.19 


.52 







Source: Developed by researcher. 

Government policy and contracting officer groups differ 
in their opinion of the utility of the methods to these type 
of contract situations. Both of these groups believe that 
parametric cost estimating methods have limited use in FPR(P) 
contract situations while the remaining groups in Table 28 
believe that they are very useful. 
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5. Fixed-Ceiling-Price With Retroactive 

Redetermination 

The results from the analysis of Fixed-Ceiling-Price With 
Retroactive Redetermination FPR(R) contract type question 
mirror those of the FPR(P) question. The composite group 
response average was 2.64 with the median value equal to 3 or 
some use. There were a large number of no responses (79) and 
the distribution was positively skewed. The control, program 
manager, and industry groups thought that the methods were 
very useful in these types of contract situations as did the 
NASA participants. These data are presented in Table 30. 

The analysis of variance for DoD component and job 
classification, presented in Table 31, highlighted not only 
the NASA group but also the cost estimators, auditors, and 
cost and price analysts. The last three groups listed 
believed, as do the contracting officers, that parametric cost 
estimating methods have only some use in FPR(R) contracting 
arrangements. The opinion of these groups is significant. 
They represent a major part (107) of the participants and a 
cross section of acquisition community who might be expected 
to be involved in applying the methods to a FPR(R) situation. 



117 



Table 30 

FPR(R) Contract Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control 

Group 


1 . 00 


3.00 


1.67 


1.50 


.82 


1.00 


2.25 


Program 

Managers 


1.00 


3.00 


2.00 


2.00 


.71 


1 . 50 


2.50 


Industry 


1.00 


4.00 


1.84 


2.00 


.96 


1.00 


3.00 


Contracting 

Officers 


2.00 


4.00 


2.76 


3.00 


.71 


2.00 


3.00 


NASA 


1.00 


2.00 


1.67 


2.00 


.58 


1.00 


2.00 


Cost 

Estimators 


1.00 


4.00 


2.67 


2.00 


1.21 


1.75 


4.00 


Cost or 

Price 

Analysts 


1.00 


4.00 


2.71 


3.00 


.90 


2.00 


3.00 


Auditors 


2.00 


4.00 


2.80 


3.00 


.71 


2.00 


3.00 



Source: Developed by researcher. 



Table 31 

Analysis of Variance For FPR(R) Contract Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


14.12 


1.57 


2.41 


.02 


Within Groups 


118 


76.76 


.65 






DoD Component 


DF 






F 


P 


Between Groups 


6 


8.87 


1.48 


2.62 


.02 


Within Groups 


108 


61.08 


.57 







Source: Developed by researcher. 
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6 . 



Fixed-Price-Incentive, Firm 



Group responses to this question were almost evenly 
divided between some use (57) and very useful (52) . The 
median response was some use and the mean response value was 
2.56. Tables 32 and 33 show the information for this 
question. 



Table 32 

FPIF Contract Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control 

Group 


1.00 


3.00 


1.63 


1.50 


.74 


1.00 


2.00 


Program 

Managers 


1.00 


3.00 


2.33 


2.00 


.82 


1.50 


2.50 


Industry 


1.00 


4.00 


1.90 


2.00 


. 97 


1.00 


3.00 


Contracting 

Officers 


2.00 


4.00 


2.64 


3.00 


. 68 


2.00 


3.00 


Cost or 

Price 

Analysts 


1.00 


4.00 


2.64 


3.00 


.90 


2.00 


3.00 


Auditors 


2.00 


4.00 


2.71 


3.00 


.69 


2.00 


3.00 



Source: Developed by researcher. 



The control and industry groups felt that the methods had 
more utility for Fixed-Price-Incentive, Firm (FPIF) contracts 
than the other groups that were surveyed. These two groups 
indicated that the methods were very useful while the 
remaining groups felt that they had only limited 
applicability. The analysis of variance data indicate that 
cost and price analysts, contracting officers, and auditors 
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(99 of the respondents) thought that parametric cost 
estimating methods had only some use. 



Table 33 

Analysis of Variance For FPIF Contract Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


16.98 


1.89 


2.97 


.00 


Within Groups 


127 


80.61 


. 64 







Source; Developed by researcher. 

7. Cost-Plus -Incentive-Fee 

Cost-Reimbursement type contracts formed a significant 
part (18%) of the contracts awarded by the DoD in 1994. [Ref 
45:p.81] Parametric cost estimating method utility in Cost- 
Plus-Incentive-Fee (CPIF) contract situations was determined 
to be somewhat useful by the survey respondents. An average 
response value of 2.62 and median of 3 was observed. Contract 
negotiators, program managers, industry respondents, and 
engineers (37 participants total) , identified in the analysis 
of variance data, thought that the methods would be very 
useful in CPIF contract situations. The remainder of the 
respondents (103) , represented in Table 34 by the contracting 
officer group, thought that the methods were of only some use 
in this type of situation. It is of interest to note the 
difference of opinion regarding the utility of parametric cost 
estimating methods in CPIF situations between the contracting 
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officers and contract negotiators. One would expect to find 
similar responses to this question from these two groups 
because of their common training and experience. Analysis of 
variance data are presented in Table 35. 



Table 34 

CPIF Contract Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control 

Group 


1.00 


2.00 


1.50 


1.50 


.54 


1.00 


2.00 


Program 

Managers 


1.00 


3.00 


1.80 


2.00 


.84 


1.00 


2.50 


Industry 


1.00 


4.00 


1.80 


1.00 


1.15 


1.00 


2.75 


Negotiators 


1.00 


2.00 


1.50 


1.50 


.58 


1.00 


2.00 


Engineers 


1.00 


2.00 


1.63 


2.00 


.52 


1.00 


2.00 


Contracting 

Officers 


1.00 


4.00 


2.71 


3 . 00 


.80 


2.00 


3.00 



Source: Developed by researcher. 



Table 35 

Analysis of Variance For CPIF Contract Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


21.03 


2.34 


3.41 


. 00 


Within Groups 


131 


89.79 


.69 







Source: Developed by researcher. 

8 . Cost-Plus-Award-Fee 

The median value of 3 reflects the choice of some use by 
55 participants to describe the utility of parametric cost 
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estimating methods to Cost-Plus-Award- Fee (CPAF) contract 
situations. The average response value was slightly lower at 
2.59, in large part due to 43 respondents selecting very 
useful as their answer to the question. The analysis of 
variance data showed a clear separation of opinion regarding 
the utility of the methods to CPAF contract situations. 
Contract negotiators, program managers, and engineers (17) 
answered that they were very useful and all of the remaining 
job classification groups (123) had the opinion that they were 
only somewhat useful. 

The analysis of variance information from the DoD 
component groups showed that the USAF had the lowest response 
average and the U.S. Army had the highest average in those 
groups. The data in Table 36 show that the USAF indicates 
that the methods are very useful and the U.S. Army only of 
some use in CPAF contract situations. Analysis of variance 
information is presented in Table 37 . The divergence between 
contracting officer and contract negotiator opinions regarding 
this contract type and parametric cost estimate utility 
observed in earlier questions is also noted here. Also 
noteworthy is the consensus that the control and engineer 
groups have on the question. They both believe that the 
methods are very useful or vital in a situation where one 
would use this type of contract. 
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Table 36 

CPAF Contract Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control 

Group 


1.00 


2.00 


1.50 


1.50 


.54 


1.00 


2.00 


Program 

Managers 


1.00 


3.00 


1.80 


2.00 


.84 


1.00 


2.50 


Industry 


1.00 


4.00 


1.80 


1.00 


1.15 


1.00 


2.75 


Engineers 


1.00 


2.00 


1.57 


2.00 


.54 


1.00 


2.00 


Contract 

Negotiators 


1.00 


3.00 


1.80 


2.00 


.84 


1.00 


2.50 


U . S . Army 


1.00 


4.00 


2.82 


3.00 


.83 


2.00 


3.25 


USAF 


1.00 


4.00 


2.13 


2.00 


.99 


1.00 


3.00 


Contracting 

Officers 


1.00 


4.00 


2.72 


3.00 


.78 


2.00 


3.00 



Source: Developed by researcher. 



Table 37 

Analysis of Variance For CPAF Contract Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


20.26 


2.25 


2.93 


.00 


Within Groups 


129 


98.98 


.77 






DoD Category 


DF 






F 


P 


Between Groups 


6 


13.76 


2.29 


3.06 


.01 


Within Groups 


117 


87 . 69 


.75 







Source: Developed by researcher. 
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9. 



Cost-Plus -Fixed- Fee 



There were no groups that displayed a consensus or 
unanimous opinion regarding the use of parametric cost 
estimating methods in situations that would also benefit from 
the use of a Cost-Plus-Fixed-Fee (CPFF) arrangement. Of the 
benchmark groups, only the control and contracting officer 
groups had a consensus of some sort on this question. Table 
38 contains only the question three benchmark groups for 
analysis . 



Table 38 

CPFF Contract Group Data 





Min 


Max 


M 


Median 


a 


Q1 


Q3 


Control 

Group 


1.00 


2.00 


1.38 


1.00 


.52 


1.00 


2.00 


Program 

Managers 


1.00 


3.00 


1.80 


2.00 


.84 


1.00 


2.50 


Industry 


1.00 


4.00 


1.65 


1.00 


1.13 


1.00 


2.00 


Contracting 

Officers 


2.00 


4.00 


2.69 


3.00 


.79 


2.00 


3.00 



Source: Developed by researcher. 



The aggregate group average response to this question was 
2.58 and a median response of somewhat useful was recorded. 
The control group had the highest regard for using the methods 
in CPFF contract situations, very useful or vital, and the 
contracting officers the lowest regard for their use in CPFF 
instances. In comparison with the average value for the 
entire database, the control, program manager, and industry 
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groups had a much higher regard for the utility of parametric 
cost estimating techniques in CPFF contracting than did any 
other group. 

10. Time & Materials 

While Time & Materials (T&M) contracts make up only a 
small percentage (6% in FY 94) of the contracts awarded by the 
DoD, a significant dollar value is contracted using this 
vehicle. [Ref 45:p.81] The majority of the responses to this 
question were divided between some use and no use in an area 
that a T&M contract might also be beneficial. The largest 
number of respondents chose some use (53) as their answer and 
49 participants chose 4, or no use to the question. The 
average response value of 3.13 and median response of 3 
indicates that the participants felt that the methods were of 
limited use in these situations. A relatively insignificant 
number (6) of people chose to respond that the methods would 
be a hindrance in situations requiring a T&M contract. 

The benchmark group data and groups identified during the 
analysis of variance are shown in Table 39. Table 40 contains 
analysis of variance results for review. With the exception 
of the program managers, there is an almost universal belief 
that the methods have little application in T&M contracting 
situations. Program staff opinion differs from the program 
managers in their belief about the utility of the methods 
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here. Program staff respondents are unanimous in their 
opinion that parametric cost estimating methods are of no use 
in T&M contract situations while program managers indicate 
that they believe that they have some use in the process. The 
groups shown in Table 39 show a belief throughout the DoD 
acquisition community that the methods are of little use in 
T&M situations. 



Table 39 

T&M Contract Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control Group 


1.00 


5.00 


3.29 


3.00 


1.50 


2.00 


5.00 


Program 

Managers 


1.00 


4.00 


2.33 


2.50 


1.21 


1.00 


3.25 


Program Staff 


4.00 


4.00 


4.00 


4.00 


.00 


4.00 


4.00 


Industry 


1.00 


5.00 


2.90 


3.00 


1.37 


2.00 


4.00 


Cost or Price 
Analysts 


1.00 


5.00 


3.27 


3.00 


1.03 


3.00 


4.00 


Contracting 

Officers 


2.00 


5.00 


3.15 


3.00 


.82 


2.75 


4.00 



Source: Developed by researcher. 



Table 40 

Analysis of Variance For T&M Contract Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


17.21 


1.91 


2.31 


.02 


Within Groups 


128 


105.89 


.83 







Source: Developed by researcher. 
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1 1 . Labor Hour 



The results of the analysis of the Labor Hour contracts, 
representing 1% of all DoD procurement actions, are similar to 
those of the T&M contracts. [Ref 45:p.81] Again, there was a 
median response of only some use and an average value of 3.13. 
Responses from the benchmark groups in Table 41 reflect a view 
that the methods have little application in this type of 
contract environment. The program staff group is unanimous in 
their opinion that the methods have no use in labor hour 
contract type situations. This differs slightly from the view 
of the program managers who see some application for 
parametric cost estimating methods in labor hour contracts. 



Table 41 

Labor Hour Contract Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control 

Group 


1.00 


5.00 


3.50 


4.00 


1.76 


1.75 


5.00 


Program 

Managers 


2.00 


4.00 


3.00 


3.00 


.71 


2.50 


3.50 


Program 

Staff 


4.00 


4.00 


4.00 


4.00 


.00 


4.00 


4.00 


Industry 


1.00 


5.00 


3.11 


3.00 


1.28 


2.00 


4.00 


Contracting 

Officers 


2.00 


5.00 


3.11 


3.00 


.87 


2.00 


4.00 



Source: Developed by researcher. 
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The analysis of the various contract types indicates that 
there are some contract types that can benefit through 
applying parametric cost estimating methods to the process 
more than others- The average values for the control, program 
manager, and industry groups indicate a relationship between 
risk, contract type, and the utility of parametric cost 
estimating methods similar to that shown in Figure 8. 
Respondents in these groups viewed the methods as having the 
most applicability in CPFF and least utility with T&M or labor 
hour contract situations. While this relationship exists, it 
may not be as pronounced as that shown in Figure 8. 
Furthermore, the relationship shows a distribution of utility 
that is skewed to the right. Figure 9 graphically displays 
the average values for the various contract types. 




Control Program Managers 

Industry Contracting Officers 

Figure 9; Contract Type Compared to Parametric Cost Estimating Utility 
Source: Developed by researcher. 
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The increase observed in the utility lines for the 
control, program manager, and industry groups is not present 
in the contracting officer line. The contracting officers may 
view risk from an individual contract standpoint rather than 
across an entire program. They may see parametric cost 
estimating as a tool to be evaluated for use in one single 
action and not having any relationship to risks in the 
contract or program. This group is also made up of a 
combination of procuring contracting officers (PCO) as well as 
administrative contracting officers (ACO) . These two 
subgroups may have different viewpoints on the utility of the 
methods based on risk and contract type that are masked when 
they are combined into one group. These differences could be 
a result of the roles both groups have in the acquisition 
process and program management cycles. 

All of the benchmark groups for this question responded 
that there was little or no application for parametric cost 
estimating methods in either T&M or labor hour contract 
situations. This type of contract is usually used in 
situations where the work can be described but not the 
quantity of work needed. Pricing for the labor or materials 
is performed by determining actual labor rates or material 
costs based on the marketplace. These two factors combine to 
make parametric cost estimating methods less efficient in 
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these situations than other estimating methods. However, the 
methods should be able to provide some use if adequate 
databases and analysis techniques are available to project the 
required quantities of labor or materials. 

D. PRE -AWARD & POST-AWARD CONTRACT ACTIONS 

Question four asks the participants to offer an opinion 
on the utility of parametric cost estimating methods to either 
pre-award actions or post-award actions. Pre-award activities 
where parametric cost estimating might be useful include 
budgeting, ICE generation, proposal analysis, or preparing for 
negotiations and price analysis. Post-award actions in which 
the methods may be useful could include evaluating an 
Engineering Change Proposal (ECP) , analyzing the cost aspects 
of a claim, negotiating a Forward Pricing Rate Agreement, or 
negotiating changes to existing contracts. These actions 
could be performed by either PCO or ACO personnel at various 
points in the acquisition process. 

1 . Pre-Award Contract Actions 

Pre-award functions such as those previously described 
are the areas in which parametric cost estimating methods are 
usually thought to apply most directly. Given this pre- 
disposition it is not surprising to find that the responses 
show a relatively high degree of utility for the methods in 
pre-award contract actions. The average response value for 
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this question is 2.36 and the median value is 2 or very- 
useful. All of the responses were coded 4 or lower indicating 
that no one felt that the methods were a hindrance in pre- 
award actions. Also, only 13 responses were coded as no use 
for the methods in pre-award applications. Tables 42 and 43 
present the group data and analysis of variance data for this 
question . 



Table 42 

Pre-Award Contract Action Group Data 





Min 


Max 




Median 


o 


Q1 


Q3 


Program 

Staff 


3.00 


3.00 


3.00 


3.00 


.00 


3.00 


3.00 


Control 

Group 


1.00 


1.00 


1.00 


1.00 


.00 


1.00 


1.00 


Corporate 

Policy 


1.00 


2.00 


1.60 


2.00 


.55 


1.00 


1.50 


Program 

Managers 


1.00 


3.00 


1.57 


1.00 


.79 


1.00 


2.00 



Source: Developed by researcher. 



Table 43 

Analysis of Variance For Pre-Award Action Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


12.16 


1.35 


1.94 


.05 


Within Groups 


161 


111.88 


.90 







Source: Developed by researcher. 
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All of the job classification groups think that 
parametric cost estimating methods are useful in pre-award 
actions but the analysis of variance data highlighted the 
program staff and program manager groups. These two groups 
have a significant difference of opinion regarding the utility 
of the methods. The program managers believe that parametric 
cost estimating methods are vital or very useful in pre-award 
situations while the program staff are unanimous in their 
belief that the methods are only of some use. The control 
group also is unanimous in their opinion. They feel that the 
methods are vital to pre-award actions. Another group of 
interest, the corporate policy group, feels that parametric 
cost estimating methods are very useful in these actions. 
This is also represented in the regression equation that 
relates business size to the responses. This equation 

Y = .351(X) - 6.56 R2 = .21 
(.15) (2.03) 

indicates that there is a negative relationship between 
business size and the response to this question. As business 
size increases the respondents believe that the methods are 
more applicable in pre-award contract actions. This might be 
expected as the larger businesses have more experience with 
using the methods than the smaller ones do. With this strong 
response, the use of parametric cost estimating methods should 
continue to be used extensively in pre-award contract actions . 
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2. Post-Award Contract Actions 

Parametric cost estimating methods are not usually 
associated with post-award contract actions. Often other cost 
or pricing analysis techniques based on costs experienced up 
to that point in the contract are used to price changes, 
modifications, claims, or other post-award actions. The 
responses to this question reflect a view that the methods 
have little use in post-award contract actions. The answers 
were almost equally divided between some use (59) and no use 
(53) producing an average response value of 3,04 and median 
response of some use. A relatively insignificant number (8) 
of respondents felt that the methods could actually hinder 
post-award contract actions. 

Analysis of variance data showed a relationship between 
the responses and the DoD component and job classification 
groups. These data are presented in Table 45. Group data are 
presented in Table 44. From the analysis of variance, it was 
noted that the control group found the methods to be vital or 
very useful in post-award contract actions as well as in pre- 
award contract actions . Program managers and NASA respondents 
agreed with the control group in believing that the methods 
were very useful in these activities. The DCAA participants 
felt that parametric cost estimating had little if any utility 
in post-award contract actions. 
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The NASA response and their long history of involvement 
with parametric cost estimating offers the potential for 
greater use in this area. This group believes that the 
methods are applicable in certain post-award contract actions 
as do the program managers and the control group. . The methods 
should provide the same benefits to a post-award negotiation 
or acquisition plan review that they do in preparing for 
negotiations in the pre-award phase or in drafting an 
acquisition plan. 



Table 44 

Post-Award Contract Action Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control 

Group 


1.00 


2.00 


1.23 


1.00 


.35 


1.00 


1.00 


Program 

Managers 


1.00 


3.00 


2.14 


2.00 


.90 


1.00 


3.00 


NASA 


1.00 


4.00 


2.50 


2.50 


.93 


1.00 


4.00 


DCAA 


2.00 


5.00 


3.75 


4.00 


.73 


3.00 


4.00 



Source: Developed by researcher. 

The DCAA group response has the highest value of any of 
the DoD component or job classification groups. Where the 
program managers are at least one level removed from 
documenting a post-award agreement, the DCAA and contracting 
officer groups are the individuals who bring it to fruition. 
Their belief that the methods are of little use in post-award 
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actions is a strong counter to the control and program manager 
groups' drive to use the methods in post-award contract 
activities . 



Table 45 

Analysis of Variance For Pre-Award Action Data 



Job Classification 


DF 






F 


P 


Between Groups 


8 


29.63 


3.70 


3.96 


.00 


Within Groups 


154 


143.95 


.94 






DoD Component 


DF 






F 


P 


Between Groups 


6 


29.60 


4.93 


5.81 


.00 


Within Groups 


140 


118.80 


.85 







Source: Developed by researcher. 

E. SUMMARY 

Questions one through four attempted to bring someone 

participating in the survey through a process of identifying 

that person' s underlying beliefs about where parametric cost 

estimating could be used. The opinions presented in these 

questions address the basic research question of: 

How could a DoD emphasis on using parametric methods in 
cost estimating affect the procurement process? 

The responses of the participants form the foundation of the 
analysis of the remaining questions as they address specific 
subsidiary research questions. 
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The remaining questions, five through ten, ask the 
respondent to address issues regarding the application of the 
methods to specific contract actions. The last five questions 
use a five point scale of agreement or disagreement with each 
statement. A response of one or two indicates agreement and 
those of three or four show disagreement. A response of five 
is coded as no opinion. 
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VII. ACQUISITION AND SOLICITATION PLANNING 



A. PURPOSE 

The purpose of this chapter is to analyze the responses 
to questions five and six. These two survey questions and the 
first four interview questions address planning areas that may 
be impacted if parametric cost estimating methods are used. 
Questions five through ten used a five point scale indicating 
the degree of agreement with a set of statements within each 
question presented. The scale ran between "strongly agree," 
a one on the scale, to "strongly disagree, " a four on the 
scale, with a five on the scale being "no opinion" regarding 
the question. For questions five through ten, a "no response" 
was coded the same as a "no opinion" response as it was 
entered into the database. 

B. METHODS, MODELS, AND SOFTWARE 

Question five is a six segment series of statements to 
which the respondents indicated that they agreed or disagreed. 
The statements are directed at answering the first subsidiary 
research question. 

What changes could offerors make to their proposal 
preparation, submission and support processes to 
accommodate using parametric methods, techniques, or 
software in estimating potential contract costs? 

Responses to each statement will be analyzed in the same 

manner used in the background questions. A discussion of each 
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statement's purpose will be followed by group data analysis 
and presentation of the data. 

1 . Acquisition Streamlining 

Question 5a asks respondents to provide their opinion 

regarding the following statement: 

Parametric cost estimating methods, models, and software 
could eventually replace traditional bottoms up cost 
estimating methods in many acquisition estimating 
applications as a result of acquisition streamlining. 

Many acquisition streamlining initiatives are based on the 

principle that business is more efficient in its use of 

resources than is the Federal Government. This statement 

addresses the potential displacement of traditional estimating 

methods by parametric cost estimating methods. The statement 

is directed at determining if acquisition streamlining will 

accelerate this process. The effects of parametric cost 

estimating replacing the traditional methods are addressed in 

later questions. 

The aggregate response to this question was one of 
indecision. An average response value of 2.49 was calculated 
from a response distribution of 1(28), 2(62), 3(67), and 

4(28). This distribution produced a median value of 3 or 
disagreement with the statement. The program staff response, 
shown in Table 4 6, is very clearly disagreeing with the 
statement. This group does not believe that traditional cost 
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estimating methods will be supplanted by parametric cost 
estimating methods. 

There are other groups shown in Table 46, however, that 
do believe that the methods could eventually replace 
traditional methods in some applications. Engineers and those 
who set corporate policy believe that parametric cost 
estimating methods could be used in many more applications as 
the primary estimating method. Greater discretion given to 
decision makers could result in the methods being applied in 
more situations. 



Table 46 

Question 5a Group Data 





Min 


Max 


M 


Median 


a 


Q1 


Q3 


Program 

Staff 


3.00 


4.00 


3.75 


4.00 


.50 


3.25 


4.00 


Engineers 


1.00 


3.00 


2.00 


2.00 


.63 


2.00 


2.00 


Corporate 

Policy 


1.00 


2.00 


1.40 


1.00 


.55 


1.00 


2.00 



Source: Developed by researcher. 



Figure 10 depicts a situation where a precise but time- 
and-cost-intensive detailed estimate and a somewhat less 
precise but quicker and less costly parametric estimate are 
compared at the same point in the acquisition cycle. [Ref 40] 
Both estimates predict the cost of the equipment will cost 
between $79 and $140. The detailed estimate provides a higher 
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probability that it will cost $105 than any other price in the 
estimate range. In some situations, the use of good business 
judgment will require a program manager or contracting officer 
to use the less precise estimate. 

In the past the acquisition community has relied heavily 
on precise, detailed estimates often contributing to program 
delays and added costs. Encouraging acquisition officials to 
make these types of decisions based on sound business 
practices is a critical part of acquisition streamlining. 
Parametric cost estimating is one of the tools available to 
analyze situations and make decisions based on good business 
judgment. 



Detailed Estunate 



P 

r 

0 

b 

a 

b 

i 

1 

i 

t 

y 




70 80 90 100 110 120 130 140 

Cost 

Figure 10: Estimate Accuracies 
Source: Shields, October 1996 
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2 . 



Information Systems Improvements 



Through statement 5b, the survey participants are asked 
to consider how technology might affect the cost estimating 
methods that are being used. 

Parametric cost estimating methods, models, and software 
could replace traditional cost estimating methods as 
larger, more compatible data bases, greater computing 
power, and improved software become available. 

Parametric cost estimating methods can be computer asset 
intensive and improvements in either hardware, software, or 
database management could have dramatic effects on their 
precision. The participants in the survey agreed with the 
statement that improvements in computer capability could allow 
parametric cost estimating methods to supplant more 
traditional methods. An average response value of 2.38 and 
median response of agreement or 2 were observed for this 
question. The corporate policy group was unanimous in their 
agreement to this statement. They had an average response of 
1.40 and standard deviation of .548 and everyone in this group 
agreed or strongly agreed. 

Technology advances could potentially change the 
characteristics for the Figure 10 parametric cost estimate to 
more closely match those of the detailed estimate. While the 
performance of parametric cost estimating systems might 
improve with computer technology, advancements there might 
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also benefit traditional methods. The same increasing 
computer capability that could make parametric cost estimating 
more precise could also be used to make bottoms up estimating 
more accurate. [Refs 20,30] Greater connectivity between data 
gathering systems could reduce the cost of -producing a 
detailed estimate just as it may make parametric cost 
estimating more accurate and faster. The corporate policy 
group response indicates greater acceptance for using 
parametric cost estimating methods in future applications, 
however . 

3 . Advantages 

The concepts in the following statement ■ provide the 
motivation for both contractor and Federal Government 
personnel to use parametric cost estimating methods in more 
areas : 

Parametric cost estimating methods, models, and software 
can save contractors and Federal Government oversight 
officials significant time and expense in developing and 
then evaluating proposals . 

The majority (142) of the survey participants agreed with this 
statement. The median response was 2 and the average response 
value observed was 2.04. The group data presented in Table 47 
show a range of activities across the acquisition community 
that believe that parametric cost estimating methods have the 
potential to speed up the acquisition process and reduce the 
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costs of proposal preparation and evaluation. The control, 
engineer, and corporate policy groups strongly feel that these 
methods can provide the advantages listed. This indicates 
that there is a willingness to use the methods on a wider 
basis. The belief and acceptance by DCAA of the potential 
that the methods could lead to greater acceptance for using 
the methods in more areas where traditional cost estimating 
methods have been used. 



Table 47 

Question 5c Group Data 





Min 


Max 




Median 


a 


Qi 


Q3 


Control Group 


1.00 


2.00 


1.27 


1.00 


.47 


1.00 


2.00 


Engineers 


1.00 


2.00 


1.45 


1.00 


.52 


1.00 


2.00 


Cost 

Estimators 


1.00 


3.00 


2.00 


2.00 


.77 


2.00 


2.00 


Corporate 

Policy 


1.00 


2.00 


1.20 


1.00 


.45 


1.00 


1.50 


DCAA 


1.00 


4.00 


2.15 


2.00 


.62 


2.00 


2.00 



Source: Developed by researcher. 



4 . Data Collection 

Question 5d asks the participants to record their 

opinions about the following statement: 

Parametric cost estimating methods, models, and software 
could significantly change the way potential contractors 
collect and summarize cost information. 
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Many cost collection systems primarily used by Federal 
Government contractors are designed or have been modified to 
collate information to support invoicing. In addition to 
providing management information, one of their secondary 
functions is to provide data for their cost estimating 
systems. Data available from these systems form the databases 
from which parametric cost estimates may be produced. This 
statement addresses the concern that these systems may not be 
ideally suited for producing information to be used in 
parametric cost estimating systems. 

The participants agreed with the statement that 
parametric cost estimating could affect the way cost data are 
collected and presented. A median value of 2 and an average 
response of 2.14 were noted. An analysis of variance 
performed on the DoD component response data showed that the 
NASA and USAF groups felt more strongly in agreement with this 
question than any of the other groups. The remaining DoD 
component groups were clustered around simple agreement with 
the statement. Within the USAF group, the USAF cost analysts 
were notable for their strong belief that the methods could 
change the way cost data were collected and summarized. Group 
response and analysis of variance data are presented in Tables 
48 and 49. 
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Table 48 

Question 5d Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


USAF Cost 
Analysts 


1.00 


2.00 


1.43 


1.00 


.54 


1.00 


2.00 


Corporate 

Policy 


1.00 


2.00 


1.20 


1.00 


.45 


1,00 


1.50 


USAF 


1.00 


3.00 


1.71 


2.00 


.59 


1.00 


2.00 


NASA 


1.00 


3.00 


1.50 


1.00 


.16 


1.00 


2.00 



Source; Developed by researcher. 



Table 49 

Analysis of Variance For Question 5d Data 



DoD Component 


DF 






F 


P 


Between Groups 


6 


12.98 


2.16 


4.05 


.00 


Within Groups 


152 


81.17 


.53 







Source: Developed by researcher. 

5 . Cost Overruns 

Most of the survey participants disagreed with the 
following statement: 

Parametric cost estimating methods, models, and software 
should decrease the probability of cost overruns where 
used. 

A total of 110 respondents disagreed with the statement which 
produced a mean response value of 2.89 and disagree (3) 
response for the median. The analysis of variance on job 
classification responses showed almost overwhelming 
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disagreement with the statement. The auditor group data in 
Table 50 represents that view. 

Two groups in the analysis of variance were noteworthy 
because of their agreement with the statement. Table 51 
contains the analysis of variance data. The program manager 
and engineer groups feel that parametric cost estimating 
methods may have some use in preventing cost overruns. 



Table 50 

Question 5e Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Auditors 


2.00 


4.00 


3.97 


3.00 


.67 


2.75 


3.00 


Engineers 


1.00 


3.00 


2.00 


2.00 


.55 


2.00 


3.00 


College 

Algebra 


2.00 


4.00 


3.00 


3.00 


. 66 


3.00 


3.00 


Program 

Managers 


1.00 


4.00 


1.86 


2.00 


1.07 


1.00 


2.00 



Source: Developed by researcher. 



Table 51 

Analysis of Variance For Question 5e Data 



Job Classification 


DF 






F 


P 


Between Groups 


9 


16.89 


1.88 


3.17 


.00 


Within Groups 


145 


85.84 


.59 







Source: Developed by researcher. 

There is also a relationship between mathematics 
education and the responses that supports the group data in 
Table 50. The regression equation 
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.02 



Y = -.094 (X) + 6.40 R2 = 

(.04) (1.66) 

indicates a weak negative relationship between the responses 
and mathematics education. The greater level of mathematics 
education that both groups have and the experience that the 
program managers have may have produced this result. These 
two groups may view the issue of cost in terms of a range of 
possible costs, similar to that presented in Figure 10, that 
parametric cost estimating methods may help to define. Given 
the flexibility to work within that range may provide a way to 
manage risks to avoid cost overrun situations. 

6. Cost and Pricing Data 

Given the response to statement 5d, one would also expect 

a similar response to the following statement: 

Parametric cost estimating methods, models, and software 
should change the way cost and pricing data submitted to 
support a proposal is viewed. 

Cost and pricing data are "... all the facts that can be 
reasonably expected to contribute to the soundness of 
estimates of future costs." [Ref 13:part 15.801] If the 
methods used to collect and summarize data are changed by a 
potential contractor then it is reasonable to assume that the 
cost and pricing data that the company submits to the 
Government in a proposal will also change. 
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The average response to this statement was agreement. An 
aggregate group average of 2.29 and a median value of 2 was 
noted. A broad cross section of the DoD acquisition community 
is presented in Table 52 who agree with this statement. 
Individuals who prepare estimates for both the Federal 
Government and the DoD contractor community are represented in 
agreement with the statement. 



Table 52 

Question 5f Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


USAF 

Contracting 

Officers 


2.00 


3.00 


2.17 


2.00 


.41 


2.00 


2.25 


Negotiators 


2.00 


2.00 


2.00 


2.00 


.00 


2.00 


2.00 


Engineers 


1.00 


3.00 


2.00 


2.00 


.50 


2.00 


2.00 


Prepare 

Estimates 


1.00 


3.00 


1.97 


2.00 


.59 


2.00 


2.00 


USAF 


1.00 


4.00 


2.06 


2.00 


.77 


2.00 


2.00 



Source: Developed by researcher. 



These groups represent individuals who would be expected 
to receive and then analyze a proposal to a solicitation which 
could include cost and pricing data. All of the groups listed 
show a consensus viewpoint. The contract negotiator group is 
also unanimous in their agreement with the statement. 

The responses to this series of statements indicate that 
the commercial industry component of the DoD acquisition 
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community is willing to use parametric cost estimating 
methods. The data also indicate that many in the Federal 
Government are willing to accept the use of the methods also. 
To take this step, both parties should agree that the 
contractor community may have to take several actions. 
Generating cost estimates based on parametric methods will 
require investments in software either for database analysis 
and the generation of models or for the purchase of existing 
models that can be modified. The benefits that might drive 
this change would accrue to both Government and industry in 
the reduced acquisition cycle times, lower cost proposal 
preparation costs, and greater management flexibility. It 
will also require some changes to the systems used to collect, 
collate, and analyze cost data within industry and the 
Government. For a contractor organization that elects to use 
parametric cost estimating methods as a primary tool for 
generating data for contract management, significant changes 
to his existing structure could be required. 

C. SOLICITATION PLANNING 

Question 6 addresses the issues raised in the second and 
third subsidiary research questions in seven parts. The 
questions it deals with are, 

1. What impacts could emphasis on using cost estimates 

based on parametric cost estimating techniques have on 
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source selection and on the evaluation of proposals by 
the DoD? and 

2 . How could source selection criteria and evaluation 
factors be constructed to fairly evaluate a proposal that 
uses a parametric estimate with one that does not? 

Providing a level playing field for all solicitation proposals 
is a primary goal of Federal Government contracting. The 
statements in this question deal with how the Government can 
construct that level playing field within the solicitation it 
issues in regard to the use of parametric cost estimating 
methods . 

1. Encouraging Parametric Cost Estimating Use 

The first statement in this question asks the respondent 

if the Government should simply encourage the use of 

parametric cost estimating methods. 

In solicitations, buying activities should encourage the 
use of parametric cost estimating for specified 
procurement actions. 

The majority of respondents overwhelmingly agreed with this 
statement, 95 individuals selected agree, and 27 selected 
strongly agree. An average response value of 2.21 and a 
median value of 2 was shown in the cumulative data. The group 
data in Table 53 is a display of groups within the DoD 
acquisition community whose responses are noteworthy for this 
question. 
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This diverse group all agree that the proposing 
organization should be encouraged to use the methods where 
applicable. As one might expect, the group representing 
industry is most emphatic in their agreement with this 
statement. Given the present environment of preference for 
disengagement from intrusive inspection systems and burdensome 
requirements, this is the most likely form of emphasis that 
DoD will use to promote parametric cost estimating techniques. 



Table 53 

Question 6a Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


DCMC Cost 
Analysts 


1.00 


3.00 


2.07 


2.00 


.59 


2.00 


2.00 


Negotiators 


1.00 


3.00 


2.00 


2.00 


.58 


2.00 


2.00 


Corporate 

Policy 


1.00 


2.00 


1.25 


1.00 


.50 


1.00 


1.75 


DCMC 


1.00 


3.00 


2.22 


2.00 


.54 


2.00 


2.00 


Prepare 

Estimates 


1.00 


4.00 


1.94 


2.00 


.77 


1.00 


2.00 


USAF 


1.00 


3.00 


1.88 


2.00 


. 62 


1.25 


2.00 



Source: Developed by researcher. 
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2. Specifying Methods, Models, or Software 

Question 6b, shown below, deals with more aggressive use 
of parametric cost estimating methods by requiring them to be 
used when the buying organization feels that they would be 
helpful. 

In solicitations, buying organizations should, specify 
parametric cost estimating methods, models, or software 
for cost proposal construction. 

The average response to this statement was to disagree. An 

average summary value was 2.73 and the median value of 3 

indicate this sentiment. The three groups whose data are 

shown in Table 54 all disagree with this statement. All three 

groups' responses are noteworthy; the control group because of 

their long study of the appropriate uses of parametric cost 

estimating methods and both contracting officer groups because 

of their experience in generating solicitations. 

Regression analyses of the GS/SES level and business size 

responses show a relationship exists. The equations and 

statistics 



GS/SES Level 

y = .286(X) - 1.45 = .10 

(.14) (1.90) 



Business Size 

Y = .471 (X) - 3.71 r2 = .03 

(. 20 ) { 2 . 12 ) 



demonstrate positive relationships between responses and 
increasing Government rank and also increasing business size. 



152 



The more senior the respondent the more they disagreed with 
the statement and the larger the business the more that they 
disagreed with the statement. It is very clear that neither 
the Government nor industry wants the Government to specify 
cost estimating methods to be used in response to a 
solicitation. 



Table 54 

Question 6b Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control 

Group 


2.00 


4.00 


3.18 


3.00 


.60 


3.00 


4.00 


USN 

Contracting 

Officers 


2.00 


3.00 


2.75 


3.00 


.45 


2.25 


3.00 


USAF 

Contracting 

Officers 


3.00 


4.00 


3.17 


3.00 


.41 


3.00 


3.25 



Source: Developed by researcher. 

3. Offeror Discretion in Using Parametrics 
Question 6c records participant opinions about the 



statement that follows; 

In solicitations, buying activities should leave 
parametric cost estimating methods, models, or software 
selection to contractor discretion. 

Leaving the use of cost estimating methods to be used 
completely up to the seller's discretion was agreed to by 111 
of the respondents. This produced a median value of 2 and an 
average response value of 2.33. The analysis of variance with 
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the DoD component data showed a relationship was present in 
the information. Of the DoD job classification groups only 
the NASA and USAF groups, 22 participants, clearly agreed with 
this statement. The data to support this are shown in Tables 
55 and 56. 

With the exception of the NASA and USAF groups, all of 
the other job classification groups were split between 
agreement and disagreement. The DCMC response data in Table 
55 is representative of these groups. Unexpectedly, there was 
also no consensus among the industry participants regarding 
this statement as there was in statement 6a. 



Table 55 

Question 6c Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


DCMC 


1.00 


3.00 


2.24 


2.00 


.60 


2.00 


3.00 


NASA 


1.00 


2.00 


1.80 


2.00 


.48 


1.50 


2.00 


USAF 


1.00 


3.00 


1.88 


2.00 


.70 


1.00 


2.00 



Source: Developed by researcher 



Table 56 

Analysis of Variance For Question 6c Data 



DoD Component 


DF 






F 


P 


Between Groups 


9 


9.77 


1.63 


2.57 


.02 


Within Groups 


144 


91.20 


.63 







Source: Developed by researcher. 
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4 . Required Cost Breakdown Formats 

Respondents to question 6d were asked to provide their 

opinions about how parametrics might change data presentation 

for solicitations through the following statement: 

In solicitations, buying activities should require a 
specific cost breakdown format for parametric cost 
estimate based proposals. 

This statement addresses the issues in the question: 

If parametric cost based estimates are used in a 
proposal, should the buyer state the format that the cost 
information is to be presented? 

With so many different systems producing estimates, some 
standardization may be helpful in evaluating proposals in a 
timely manner. A specified cost breakdown format is one 
method of standardization that is available. 

The majority (108) of the respondents agreed to this 
premise producing an average response value of 2.35 and median 
response of 2. Analysis of variance on the job classification 
responses showed that a relationship between the responses and 
the classifications was present. Tables 57 and 58 contain the 
data pertinent to the question 6d statement. The auditor, 
program staff, program manager, and contract negotiator 
groups, 54 respondents total, all indicated agreement with 
this statement. These groups might all be part of a source 
selection team and see a value in having a specified format. 
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Another significant group in the source selection process, the 
contracting officers, did not come to a consensus on either 
agreement or disagreement with the statement, however. 



Table 57 

Question 6d Group Data 





Min 


Max 




Media 

n 


a 


Q1 


Q3 


Auditors 


1.00 


3.00 


2.06 


2.00 


. 65 


2.00 


2.25 


Negotiators 


1.00 


2.00 


1.67 


2.00 


.52 


1.00 


2.00 


DCAA 


1.00 


4.00 


2.06 


2.00 


. 63 


2.00 


3.00 


Contracting 

Officers 


1.00 


4.00 


2.50 


2.00 


.73 


2.00 


3.00 


Program Staff 


1.00 


3.00 


2.00 


2.00 


.71 


1.50 


2.00 


Program 

Managers 


1.00 


3.00 


1.86 


2.00 


.69 ' 


1.00 


2.00 



Source: Developed by researcher. 



Table 58 

Analysis of Variance For Question 6d Data 



Job Description 


DF 






F 


P 


Between Groups 


9 


11.19 


1.24 


2.05 


.04 


Within Groups 


159 


96.59 


. 61 







Source: Developed by researcher. 

Regression analysis with the GS/SES level and business 
size group data showed positive relationships in the data. 
The equations 

GS/SES Level 

Y = .300(X) - 1.91 = .10 

(.14) (2.01) 
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Business Size 

Y = .484 (X) - 4.61 r2 = .25 

(.18) (2.45) 

describe these relationships. Again, as one's rank or 
business size increases so does the response value to the 
question. These results indicate that experience with the 
methods would lead to allowing the seller discretion in not 
only their use but in how they choose to present the data from 
those estimates. 

5. Cost and Technical Proposal Relationships 

A second area of concern in evaluating proposals is that 

of having the information in each section correspond to one 

another. The following statement addresses this issue: 

In solicitations, buying activities should require a 
specified format to extract labor hours, subcontractor, 
or material quantity estimates for technical proposal 
evaluation. 

A standard format required by the buyer stating that format 
could provide that concurrency. Similar results to the 6d 
statement were observed. An average response value of 2.37 
and a median value indicate agreement with the statement by 
the entire group of participants. The group of contract 
negotiators had a consensus that they strongly agreed with 
this statement. Their group response was the only group 
response to reach any type of consensus of all of the 
demographic categories identified. 
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6 . 



Statistical Evaluation Criteria 



Determining what criteria will be used to evaluate the 

validity of the CERs used to produce estimates included in a 

proposal and stating them in the solicitation is another 

method of creating a level playing field. Statement 6f, as 

shown below, examines this area: 

In solicitations, buying activities should state criteria 
used to evaluate the statistical significance of cost 
estimating relationships used to support a parametric 
cost estimating model. 



A great deal of agreement was noted for this statement. The 
average response value was 2.06 and the median value was a 2, 
largely a result of 137 respondents electing to. either agree 
or strongly agree with the statement. The group data in Table 
59 show a wide range of acquisition activities across the 
entire acquisition community who agree with the statement. 



Table 59 

Question 6f Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Program Staff 


1.00 


2.00 


1.50 


1.50 


.58 


1.00 


2.00 


Cost 

Estimators 


1.00 


3.00 


1.97 


2.00 


.55 


2.00 


2.00 


Prepare For 
Negotiations 


1.00 


4.00 


2.00 


2.00 


.74 


2.00 


2.00 


Prepare 

Estimates 


1.00 


3.00 


2.03 


2.00 


. 65 


2.00 


2.00 


USAF 


1.00 


4.00 


2.24 


2.00 


.66 


2.00 


2.25 



Source : 



Developed by researc 



her. 
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Information regarding CER validity could be interpreted to be 
required by FAR part 15.406-5 in that it could be defined as 
a significant factor that might be considered in awarding a 
contract . 

7. Traditional or Parametrics Methods in Solicitations 

Statement 6g, show below, asks survey participants their 

opinion about the concept of fairness in a solicitation if 

parametric methods are either specified or excluded. 

In solicitations, buying activities should require cost 
proposals based on either parametric cost estimating 
methods or traditional methods. 

A source selection plan should not favor one proposal over 
another simply by the way the proposal evaluation plan is 
constructed. The plan must make sure procedures do not act 
to exclude or discourage potential offerors from submitting 
proposals. The statement addresses the problem of fairly 
evaluating a parametric-based cost proposal with one produced 
using more traditional detailed methods. Can an evaluation 
plan be constructed that will fairly evaluate both types of 
cost estimates or must the buyer specify which an offeror 
should use because they cannot be evaluated fairly? 

There was no agreement on this question at the level of 
the entire pool of participants nor on an individual group 
level . The aggregate response average value was 2 . 55 and a 
median value of 3 was observed. These data indicate that the 
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group disagreed in general with the statement but that they 
were effectively split over agreeing and disagreeing. There 
were also a significant number (51) of no opinion or no 
responses to this question. 

D . SUMMARY 

The results from the analysis of the statements in 
questions five and six provide a number of areas that could 
be affected by the use of parametric cost estimating methods. 
Chief among these areas is the contractor cost estimating 
system. The systems will respond to the encouragement that 
the Federal Government members of the DoD acquisition 
community provides . There was universal agreement that buying 
activities should encourage the use of parametric cost 
estimating methods but not require them. 

One of the most effective methods of encouraging the 
use of the methods is in the source selection process. An 
evaluation of a parametric-based proposal that is viewed by 
the offeror to be unfair could be taken as discouragement. 
The trend should remain with telling the offerors what is 
required and letting them propose methods of achieving that 
goal. In that regard, source selection plans should be 
constructed to allow any reasonable type of cost proposal 
submitted to be fairly evaluated. 
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Required formats for data submission in terms of the 
cost proposal or technical proposal were viewed by the group 
as useful in creating a level playing field. These actions 
have the advantage of letting the offerors meet the goal that 
is required rather than telling them how it is to be 
accomplished. They prevent the Government from dictating 
specific methods to be used while every offeror has some idea 
of how the information provided will be used. They have the 
added benefit of providing some consistency throughout the 
proposals which speeds up the evaluation process. 

If parametric estimating methods are acceptable for 
use as a basis for a cost proposal, the proposal must state 
how they will be evaluated when compared with other methods. 
FAR part 15.605(c) (d) (1) addresses the issue of evaluation 
factors under which this issue falls. 

... at a minimum, the solicitation shall clearly 
state the significant evaluation factors, such as 
cost or price, cost or price-related factors, past 
performance and other non-cost or non-price-related 
factors, and any significant subfactors, that will 
be considered in making the source selection. [Ref 
13:part 15.605] 

A buying organization must consider constructing their 
solicitation to include the possibility of parametric 
estimate-based cost proposals if all proposals are to be 
evaluated fairly. The members of the source selection 
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evaluation board should also consider this issue as cost is 
always an evaluation factor. 
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VIII. PROPOSAL ANALYSIS AND NEGOTIATIONS 



A. PURPOSE 

Chapter VIII is an analysis of the responses to questions 
seven through ten. These survey questions and the last four 
interview questions address issues that may arise during the 
evaluation of a parametric-based proposal and subsequently the 
negotiations that may result. The four survey question 
responses were graded along the same five point scale used in 
questions five and six. With the exception of question nine, 
all of the questions were a series of statements upon which 
participants recorded their opinions. 

B. COST PROPOSAL SUPPORTING DATA 

Question seven is a three part series of statements that 
again address the first subsidiary research question. 

What changes could offerors make to their proposal 
preparation, submission, and support processes to 
accommodate using parametric methods, techniques or 
software in estimating potential contract costs? 

In the three statements for question seven, an offeror has 

decided to provide a parametric-based cost estimate in 

response to a solicitation. The question addresses the 

information that might be submitted to substantiate that 

estimate. The Government goal in obtaining information about 

the estimate centers around regenerating that estimate to 

understand how it was generated, its limitations, and 
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strengths. Providing data to do this is a vital part of 
submitting a successful proposal. 

1. Software and Input Data 

Statement 7a addresses one of the ways that an offeror 
could chose or be required to provide supporting information 
for a cost proposal. 

A potential contractor should provide the parametric cost 
estimating model software and input data used to support 
a cost proposal. 

Providing the software and input data used to produce an 
estimate is one way to meet the Government goal of being able 
to reproduce the estimate. Without these tools it cannot be 
accomplished. There was a strong agreement to this statement 
in the responses. The response average was 1.81, generated in 
large part by the large number of people who answered strongly 
agree (66) and agree (95) to this statement. Responses to 
this statement also had a small standard deviation (.76) 
indicating a consensus among the respondents on this 
statement. The group data in Table 60 provides a sample of 
the groups that agreed to this statement within the DoD 
acquisition community. 

Agreement to this statement covered the entire 
spectrum of functional areas that might be tasked to evaluate 
a cost proposal. All responded that this information should 
be provided by an offeror in support of a cost proposal. The 
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industry response to the statement was not as clear. There 
was no consensus in the industry response as a whole nor in 
the group that set corporate policy. 



Table 60 

Question 7a Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Auditors 


1.00 


4.00 


1.65 


2.00 


.70 


1.00 


2.00 


Cost 

Analysts 


1.00 


4.00 


1.73 


2.00 


.72 


1.00 


2.00 


Contracting 

Officers 


1.00 


3.00 


1.83 


2.00 


.54 


1.50 


2.00 


Cost 

Estimators 


1.00 


3.00 


1.83 


2.00 


.72 


1.00 


2.00 


Engineers 


1.00 


3.00 


1.91 


2.00 


.70 


1.00 


2.00 



Source: Developed by researcher. 



2. Justifying Parametric Use 

Beyond providing the software and input data, there 
should be a supportable reason to use parametric cost 
estimating methods. They should be applicable to the 
procurement action in question and the work relatable to the 
databases used. Shown below, statement 7b examines this 
issue . 

A potential contractor should provide justification to 
support the use of parametric cost estimating models and 
input data in his cost proposal. 

The majority of the participants answered that they agreed 

(107) or strongly agreed (64) with the statement. The average 

response value was 1.75, a standard deviation of .65 was noted 
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and agree was the median response. The widespread agreement 
to this statement includes almost every functional group in 
the DoD acquisition community. The major functional areas of 
the community that typically are involved in evaluating 
proposals are shown from buying organizations to the industry. 
All of the groups shown in Table 61 have a consensus on the 
issue. 



Table 61 

Question 7b Group Data 





Min 


Max 


M 


Median 


a 


Q1 


Q3 


Control Group 


1.00 


3.00 


1.73 


2.00 


. 65 


1.00 


2.00 


Cost Analysts 


1 . 00 


3.00 


1.73 


2.00 


. 60 


1.00 


2.00 


Contracting 

Officers 


1.00 


3.00 


1.76 


2.00 


.54 


1.00 


2.00 


Cost 

Estimators 


1 . 00 


2.00 


1 . 67 


2.00 


.49 


1.00 


2.00 


Engineers 


1.00 


3.00 


2.00 


2.00 


.74 


1.25 


2.75 


DCAA 


1.00 


4.00 


1.71 


2.00 


. 64 


1.00 


2.00 


DCMC 


1.00 


3.00 


1.86 


2.00 


. 64 


1.00 


2.00 


Industry 


1.00 


4.00 


1.70 


2.00 


.82 


1.00 


2.00 



Source: Developed by researcher. 



3. Data and Rationale 



Additional information that is critical to understanding 
a parametric cost based estimate is knowing how it has been 
validated for use. Statement 7c is presented below followed 
by the analysis of the data collected regarding this issue. 
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A potential contractor should provide data and rationale 
used to validate a parametric cost estimating model for 
a specific set of circumstances in support of a cost 
proposal . 

The statement indicates the offeror should provide information 
about how the model was modified, the database updated, and 
how often these tasks were performed to give the buyers the 
level of understanding that they need. The groups shown in 
Table 62 again show the level of agreement with this statement 
across the DoD acquisition infrastructure. 



Table 62 

Question 7c Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Program Staff 


2.00 


2.00 


2.00 


2.00 


.00 


2.00 


2.00 


Control Group 


1.00 


3.00 


1.73 


2.00 


.65 


1.00 


2.00 


Cost Analysts 


1.00 


3.00 


1.66 


2.00 


.52 


1.00 


2.00 


Contracting 

Officers 


1.00 


3.00 


1.68 


2.00 


.52 


1.00 


2.00 


Cost 

Estimators 


1.00 


2.00 


1.67 


2.00 


.49 


1.00 


2.00 


Engineers 


1.00 


3.00 


1.75 


2.00 


. 62 


1.00 


2.00 


DCAA 


1.00 


4.00 


1.57 


2.00 


. 63 


1.00 


2.00 


DCMC 


1.00 


3.00 


1.79 


2.00 


. 63 


2.00 


3.00 


Industry 


1.00 


4.00 


1.65 


2.00 


.75 


1.00 


2.00 


NASA 


1.00 


3.00 


1.75 


2.00 


.54 


1.00 


2.00 


U.S. Navy 


1.00 


2.00 


1.67 


2.00 


.48 


1.00 


2.00 


USAF 


1.00 


2.00 


1.71 


2.00 


.47 


1.00 


2.00 



Source: Developed by researcher 
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Agreement to this statement was also strong and widely 
represented in the DoD community. Strongly agree was selected 
by 71 and agree by 106 respondents leading to an average 
response value of 1.67, a small standard deviation of .61 and 
a median value of 2. 

Offerors should provide information described in the 
three statements in question seven. Gathering these data 
could require new information systems to be used in industry 
to capture, collate, and display this information. The 
responses show support from industry to provide this 
information as well as a desire by those charged with 
evaluating proposals to have these data available. 

C . DCAA SUPPORT 

After a proposal is submitted, it is then analyzed in 

accordance with the evaluation plan. In a four part series of 

statements, question eight addresses possible functions that 

may need to be performed and who is best suited to do them. 

These statements address issues raised in the second 

subsidiary research question. 

What impacts could emphasis on using cost estimates based 
on parametric estimating techniques have on source 
selection criteria and on the evaluation of proposals by 
the DoD? 

Each statement assumes that DCAA is the best organization to 
perform the task identified. The respondents are asked to 
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fill in another organization if they feel that DCAA is not the 
appropriate action agency for the function in question. 
Information about the written responses will be provided with 
the analysis for each question. 

1. Establishing Validity Metrics 

Determining the validity of a parametric model to a given 
set of circumstances is critical to understanding the outputs 
of that model, their uses, and limitations. Statement 8a, 
shown below, addresses this concern. 

DCAA. should establish metrics to determine a parametric 

cost estimating model's validity in a given situation. 

Defining measurement criteria to gauge the effectiveness of 
a model in a particular situation refers to the concept of 
calibration defined in the Parametric Cost Estimating 
Handbook. If models are modified for use, descriptions of how 
they were changed to reflect the situation in which they will 
be used should be provided to proposal evaluators. This 
statement asks the respondent to determine who should set 
forth these criteria. 

There was a large amount of agreement with this statement 
indicating that DCAA should perform this function. The 
average response value of 2.25 results from the large number 
of participants (121) agreeing with the statement. The 
program staff, U.S. Army cost analyst, and USAF contracting 
officer groups represent agreement within the survey 
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participants. This is contrasted with the disagreement that 
the control group expressed. The statement group data are 
shown in Table 63. 



Table 63 

Question 8a Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Program Staff 


2.00 


2.00 


2.00 


2.00 


.00 


2.00 


2.00 


Control Group 


2.00 


4.00 


2.73 


3.00 


.77 


2.00 


3.00 


U . S . Army 
Cost Analysts 


1.00 


3.00 


1.93 


2.00 


.46 


2.00 


2.00 


USAF 

Contracting 

Officers 


1.00 


2.00 


1.83 


2.00 


.41 


1.75 


2.00 



Source: Developed by researcher. 



Regression analysis also showed a disagreement in the 
responses. The business size regression equation 



Business Size 

Y = .733{X) - 7.64 = .51 

(.17) (2.36) 



indicates that as the business size increases so does the 



disagreement with the statement. The same analysis performed 
with the GS/SES level data shows an opposite relationship. 
The equation and data 
GS/SES Level 

Y = -.418(X) + 8.28 R2 = .23 

(.13) (1.84) 

show a negative relationship. As the Government rank rises so 
does the level of agreement with the statement. The division 
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is clearly along a Government- Industry split in the 
acquisition community. 

There are two areas that may be the source of the 
disagreement. The respondents may feel that the function does 
not need to be performed or that DCAA. is not the appropriate 
organization to do the work. Since the functions described 
appear as a central concept in the Parametric Cost Estimating 
Handbook and several papers written by members of the control 
group, the likely source of the disagreement is in the area of 
who should do the work. Two answers were written in by the 
control group. Three people recommended that either 

1. DCMC should perform this function, (2) or that 

2. This work should be left to the contractor's 
discretion. (1) 

These two responses were also prominent in the aggregate of 
written responses to this statement. 

In all of the surveys returned, six groups other than 
DCAA were identified to perform this function: 

1. Buying Organizations (14) 

2. Offerors (12) 

3. DCMC (15) 

4. Cost Analysis Improvement Group (CAIG) (2) 
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5. A DQIC, DCAA, Buying Organization, and Offeror Team 
( 2 ) 

6. Program Managers (1) 

The majority of the added recommendations concentrated on 
DCMC, the buying organization, or the contractor performing 
the work. DCAA and the control group recommend leaving this 
to the contractor' s discretion rather than assigning it as a 
DCAA responsibility. DCMC and the buying organizations, USAF, 
U.S. Army, and U.S. Navy, would prefer to take on this 
function rather than task DCAA with it. 

Both possible issues of contention in this statement are 
addressed by Dennedy, Deutsch, and Hertling in their article. 
They state that this function must be accomplished and 
recommend that it be done prior to using the models to 
generate estimates. [Ref 8:p.l] They also directly address 
the topic of who should perform this verification activity for 
models generated by the potential offeror. 



The parametric model (or CER model) should be 
developed at the local level in coordination with 
the cognizant ACO and DCAA auditors. Ideally, a 
teaim made up of ACO pricing, buying office 
representatives and the contractor' s parametric 
model builders should develop the model jointly in 
an integrated product team (IPT) process. The 
joint team should review and validate the database 
upon which the model is based, the logic and 
statistical validity of the model itself and how 
the model will fit into the contractor' s estimating 
system. DCAA, as an independent examiner, will 
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provide input on the contractor's controls, 
database, and updating procedures, [Ref 8: p.l] 

In this way, the teaming concept and IPTs are introduced at a 
very early point in the procurement process. The write-in 
responses indicating that a team would be beneficial came from 
both DCAA. and DQSIC for this segment of question eight. While 
teaming may be the best solution to the problem of who should 
validate and how a model should be validated, the buying 
organizations do not seem to have considered that idea at this 
point in responding to this statement. 

2. Updating Models 

A second facet of model validation concerns when the 
model is modified or database updated and who should make this 
decision. Survey participants were asked to record their 
opinions to statement 8b: 

DCAA should establish criteria to determine when a 
parametric cost estimating model should be updated. 

The same basic issues that are evident in the concept of 

calibration apply to when a model should be calibrated. The 

participants agreed to this statement in the majority; 126 

people either strongly agreed or agreed with the statement. 

An average response value of 2.20 and median response of 2 was 

observed. The data shown in Table 64 represent the agreement 

throughout the acquisition community on this issue. The 

unanimous agreement in the program staff group and degree of 
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consensus reached on this issue in the other groups is 
noteworthy. Industry and Government members of the 
acquisition community agree that someone should set criteria 
to determine when models should be updated. A significant 
portion (20%) of the respondents recommended that someone 
other than DCAA be tasked with this function. 



Table 64 

Question 8b Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Program 

Staff 


2.00 


2.00 


2.00 


2.00 


.00 


2.00 


2.00 


Control 

Group 


1.00 


3.00 


2.00 


2.00 


.63 


2.00 


2.00 


U . S . Army 
Cost 

Analysts 


1.00 


3.00 


2.00 


2.00 


.52 


2.00 


2.00 


Industry 

Cost 

Analysts 


1.00 


2.00 


1.86 


2.00 


.38 


2.00 


2.00 


Contracting 

Officers 


1.00 


3.00 


2.00 


2.00 


.61 


2.00 


2.00 



Source: Developed by researcher. 



With the exception of the program managers, the list of 
other recommendations is the same as in the previous 
statement. The buyer (9), offeror (5), DCMC (18), and a team 
arrangement (4) were the groups most often recommended to 
perform this work. Of the responses from buying 
organizations, the write-in responses recommended either that 
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they should determine when a proposed model requires updating 
or that DCMC take on that work. The DCAA written 
recommendations indicate that the offeror should perform this 
work. 

The Dennedy et al article does not specifically address 
the issue of when a model should be updated but the concept is 
included in the duties listed for the team members. Inherent 
in the article is an assumption that the offeror has taken the 
initiative to keep the model current and that it is applicable 
to the given situation. The Government functions in a review 
and evaluation mode with three organizations, DCAA, DCMC, and 
the buying office, performing various review .duties. The 
currency of the model is considered in the: 

1 . DCAA review of calibration and validation of the 
tools, 

2. DCMC technical review of the contractor's cost 
modeling technique, and 

3. Buying office review of the offeror's assumptions 
made in the model. [Ref 8:p.3] 

Ideally, this would be performed in the team environment 
presented in the previous statement analysis. However, the 
written responses to this question indicate that the buying 
organizations, DCMC and DCAA, do not view teaming as an answer 
to this problem. 
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3. 



Model Validation 



The issues addressed in statement 8c, shown below, are 

centered around the concept of model validation. 

DCAA should verify that a parametric cost estimating 
model accurately predicts costs for the expected contract 
work. 

While there is no way to be completely certain, until after 
the work is done, that a model has accurately predicted the 
costs for a project, there are methods that can be used to 
determine its accuracy. This process is described in detail 
as validation in the Parametric Cost Estimating Handbook. It 
may include: 

1. Calibration of the model to historical cost data, 

2. Estimating the cost of past completed projects, and 

3. Comparing these estimates with actual costs to 
demonstrate acceptable accuracies, [Ref 37:p.l8] 

as steps in a validation process. The average response value 
of 1.93 and median value of 2 show a strong level of agreement 
with the survey statement. The strong support for the 
statement is shown throughout the survey respondents in the 
group data presented in Table 65. Program personnel, 
contracting officers, and industry groups all agreed that DCAA 
should perform this verification. That a smaller number of 
written recommendations was received for this statement 
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indicates agreement that DCAA is also the organization best 
suited for doing this work. 



Table 65 

Question 8c Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Program 

Staff 


2.00 


2.00 


2.00 


2.00 


. 00 


2.00 


2.00 


U . S . Army 
Cost 

Analysts 


1.00 


4.00 


1.94 


2.00 


. 68 


2.00 


2.00 


Industry 

Cost 

Analysts 


1.00 


3.00 


1.86 


2.00 


. 69 


1.00 


1.00 


Contracting 

Officers 


1.00 


3.00 


1.83 


2.00 


.59 


1.00 


2.00 


Program 

Managers 


1.00 


2.00 


1.83 


2.00 


.41 . 


1.75 


2.00 


Corporate 

Policy 


1.00 


2.00 


1.75 


2.00 


.50 


1.25 


2.00 


Prepare For 
Negotiations 


1.00 


4.00 


1.83 


2.00 


.67 


1.00 


2.00 



Source: Developed by researcher. 



Dennedy, Deutsch, and Hertling also state clearly 
the function should be performed by DCAA but they emphasize 
that ideally it would be done in a team environment. [Ref 
8:p.3] The results of the recommendations received show that, 
again, buyers would prefer to retain this work or task DCMC 
with it. DCMC written recommendations are that they would 
prefer to retain it. It should be noted that only 15% of the 
respondents chose to provide a written response. Of the total 
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number of written recommendations to this statement, DCMC 
provided six. DCAA is well-suited to perform this task in 
their traditional role, and the lack of DCAA written responses 
shows an agreement to take on this work. 

4 . Model Operation 

The basis of analyzing a cost proposal is understanding 
the proposal in detail and the relationships within it. Using 
an offeror's model to generate cost estimates is one method of 
applying the information provided about that model to ensure 
this level of understanding. The final statement of question 
eight asks the respondents to give their opinion about the 
issues in this statement: 

DCAA should run a contractor' s parametric cost estimating 

model for purposes of analyzing the cost proposal. 

There was again a great deal of agreement with DCAA being 
tasked with operating an offeror's model for these purposes. 
There were 129 responses in agreement with this statement 
which produced an average response value of 2.08 with a .74 
standard deviation and a median value of 2. Table 66 presents 
the group data for this statement. Notably, the control, 
program manager, and program staff groups were unanimous in 
their agreement with the idea that DCAA should provide this 
service. One major buying organization, contracting officers 
and three job description groups reached a consensus that this 
service should be provided. 
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Table 66 

Question 8d Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Control 

Group 


2.00 


2.00 


2.00 


2.00 


.00 


2.00 


2.00 


Program 

Staff 


2.00 


2.00 


2.00 


2.00 


.00 


2.00 


2.00 


U . S . Army 
Cost 

Analysts 


1.00 


3.00 


2.00 


2.00 


.56 


2.00 


2.00 


Contracting 

Officers 


1.00 


3.00 


1.95 


2.00 


.52 


2.00 


2.00 


Program 

Managers 


2.00 


2.00 


2.00 


2.00 


.00 


2.00 


2.00 


U . S . Army 


1.00 


4.00 


2.03 


2.00 


.58 


2.00 


2.00 


Prepare For 
Negotiations 


1.00 


4.00 


1.93 


2.00 


.63 


2.00 


2.00 


Prepare 

Estimates 


1.00 


4.00 


2.11 


2.00 


.64 


1.00 


2.00 



Source; Developed by researcher. 



Written responses to this statement show that buyers and 

DCMC would prefer to perform this work themselves. The lack 

of DCAA written responses would indicate that they agree with 

their organization being tasked with this function. 

Regression analysis performed on the experience data presents 

the following relationship: 

Y = -.192 (X) + 7.48 R2 = .04 

(.07) (2.06) 

The weak negative relationship indicates that as experience 
increases so does the agreement with this statement. Stronger 
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support for this statement is coming from more experienced 
groups such as the prepare estimates work classification 
group. Notably there was a lack of consensus in the industry 
and corporate policy groups on the issue of DCAA operating a 
parametric cost estimating model for analysis purposes. 

The Dennedy article provides a description of a possible 
mechanism for accomplishing an agency running an offeror's 
model. The article espouses having the offeror make the model 
available to the Government rather than providing a copy of 
the model to the Government as discussed in question 7a. The 
agency responsible for operating the model in the contractor 
facility is not addressed, however. The chief- advantage to 
this solution is the model remains a part of the estimating 
system that generated the offer in the proposal rather than a 
separate piece to be inserted into a negotiating position. 

5 . Summary 

Agreement was observed for all four of the question eight 
statements. Minor areas of disagreement seem to be related to 
who would perform the work or function described rather than 
if the work should be performed at all. At most, 25% of the 
respondents chose to offer a recommendation for some 
organization other than DCAA to be tasked with the work 
described in any one of the statements. Additionally, less 
than 25% of the respondents disagreed with any one of the 
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statements in question eight. Written responses to these 

statements indicate a preference for two groups, DCMC and the 

buying organizations, taking on this function if DCAA does not 

do it. This desire to be involved in these aspects of the 

model evaluation could be beneficial. The team environment 

described by Dennedy, Deutsch and Hertling is a viable vehicle 

for combining the experience, knowledge, and needs of these 

groups to provide a single face to industry in another area of 

the acquisition process to address common goals. 

D. TECHNICAL PROPOSAL EVALUATION IMPACTS 

Participants to question nine, shown below, were asked to 

record their opinions about the information that parametric 

methods could provide in technical proposal evaluation. 

Construction of cost estimating relationships used in 
parametric cost estimate based proposals should be used 
as a factor in evaluating a contractor's level of 
understanding of the technical requirements in a 
solicitation. 

Subsidiary question two and three issues are addressed in this 
statement. Source selection criteria and evaluation factors 
should take the entire proposal into account. Technical 
understanding and capability can be revealed in the way the 
offeror intends to manage the effort, the way the 
relationships in the cost estimate are related as well as in 
the verbiage in the technical proposal itself. 
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The average response value of 2.37 and median value of 2 
indicate a fair amount of agreement with this statement. 
Table 67 shows that the program manager and NASA groups were 
strongest in their agreement with the statement. Buying 
organizations, U.S. Navy, support groups in the acquisition 
community, and contracting officers, particularly USAF 
contracting officers, also agreed with this premise. 

Table 67 

Question 9 Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Contracting 

Officer 


1.00 


4.00 


2.18 


2.00 


. 68 


2.00 


2.00 


USAF 

Contracting 

Officers 


2.00 


2.00 


2.00 


2.00 


.00 ■ 


2.00 


2.00 


Cost 

Estimators 


2.00 


4.00 


2.28 


2.00 


.49 


2.00 


2.25 


NASA 


1.00 


2.00 


1.86 


2.00 


.38 


2.00 


2.00 


U.S. Navy 


1.00 


3.00 


2.00 


2.00 


.56 


2.00 


2.00 


Program 

Managers 


1.00 


2.00 


1.67 


2.00 


.52 


1.00 


2.00 


Auditors 


1.00 


4.00 


2.13 


2.00 


.73 


2.00 


3.00 


Technical 

Support 


2.00 


4.00 


3.67 


4.00 


1.5 


2.00 


4.00 



Source: Developed by researcher. 



Analysis of variance of job description groups revealed 
that program managers, contracting officers, and auditors, 69 
respondents, agreed that CER construction should be considered 
as a technical evaluation factor. It also showed that the 
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technical support participants strongly disagreed with this 
statement although they do represent a very small group within 
the pool of respondents. This is an interesting response from 
those who would be likely to be assigned technical proposal 
evaluation duties. Table 68 contain the job classification 
analysis of variance data. 



Table 68 

Analysis of Variance For Question 9 Data 



Job Description 


DF 






F 


P 


Between Groups 


9 


14.51 


1.61 


2.51 


. 01 


Within Groups 


134 


86.03 


.64 







Source: Developed by researcher. 

Weak negative relationships between responses and both 
GS/SES level and experience were also noted. The equations 
GS/SES Level 

Y = -.336(X) + 7.29 = .13 

(.15) (2.07) 

Experience 

Y = -.179(X) + 7.45 R2 = .02 

(.09) (2.44) 

indicate that as GS/SES level or experience increases the 
agreement with the statement also increases slightly. This 
also supports the agreement levels seen in the auditor, cost 
estimating, and NASA groups. These groups are among the more 
experienced groups of all the demographic categories in the 
respondent pool. 
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The agreement with question nine indicates that 
consideration should be given to this aspect of technical 
proposal evaluation. With the strong disagreement from the 
technical support community, it may be prudent to involve 
someone from this community early in the process of developing 
an evaluation plan to encourage a team effort. Consideration 
should also be given during construction of the plan to 
allowing flexibility in the event both parametric-based and 
traditional cost proposals are submitted. As discussed in 
question 6g, fairness is essential in any source selection 
plan. 

E . NEGOTIATIONS 

Question ten is a four part series of statements dealing 

with the effects that parametric cost estimating may have on 

the negotiation process. This process can be a factor in both 

pre-award as well as post-award circumstances as noted in 

question four. Issues in the fourth subsidiary research 

question are addressed in this series of statements. 

1. Cost or Price Negotiation Criticality 

The first area of interest in statement 10a, shown below, 

is the effect estimate accuracy might play in negotiations. 

Using parametric cost estimating methods should make 
negotiating costs less critical as cost estimate accuracy 
improves . 
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As estimates become more precise, will the issue of cost 
become less of an issue? The question assumes that cost will 
always remain an evaluation factor in any proposal evaluation. 
There was a lack of agreement and consensus among the 
participants on this issue. An average response value 
indicates disagreement but the median value of 2 indicates 
agreement with the statement. The program staff and industry 
cost analyst data displayed in Table 69 are the only 
demographic categories that exhibit any consensus on this 
issue. The program staff group is unanimous in their 
disagreement that costs will become less important as estimate 
accuracy improves. The industry cost analysts see a shift in 
emphasis from costs to other factors as estimates become more 
precise . 



Table 69 

Question 10a Group Data 





Min 


Max 


y- 


Median 


a 


Q1 


Q3 


Program 

Staff 


3.00 


3.00 


3.00 


3.00 


.00 


3.00 


3.00 


Industry 

Cost 

Analysts 


2.00 


3.00 


2.14 


2.00 


.38 


2.00 


2.00 



Source: Developed by researcher. 



FAR 15.406-4 notes that cost will be included as a factor 
in source selection. This statement raises the question of 
how much will cost be a factor, if because of increasing 
accuracy, the acceptable range of both parties in a 
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negotiation is very narrow and overlapping. Parametric 
estimates derived from the same models and estimating systems 
for both offeror and buyer have the potential for this 
situation if the offeror' s facilities are used as discussed in 
the statement 8d analysis. In the event that this occurs, the 
negotiations may shift from cost concerns to ones of a 
technical ability or schedule nature. Another possibility 
that is more likely is a shift in negotiations to inputs to 
the model and the assumptions that go with them if the model 
and estimating system are accepted as validated by both 
parties in a negotiation situation. 

2. Cost as an Independent Varic±>le Impacts 
Statement 10b attempts to relate Cost As An Independent 
Variable (CAIV) with parametric cost estimating during the 
negotiation process. The second statement of question ten is 
shown as : 

Using parametric cost estimating methods should have 
little effect on negotiations as the concept of cost as 
an independent variable shifts emphasis to technical and 
schedule concerns for a given value. 

The concept of CAIV shifts emphasis to technical or other 
issues by making costs more of a constraint than an outcome of 
a series of events in a program. Prior to CAIV, there was an 
emphasis on cost control but it was still managed as a 
dependent variable in the program life cycle. CAIV introduced 
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the idea that costs would actually be controlled to some 
degree by working within some trade space with technical 
capability, logistics supportability, schedule, and other 
concerns to produce and field affordable weapons systems in a 
timely manner. 

There was a large degree of disagreement with this 
statement. The majority of respondents to this question (104) 
either disagreed or strongly disagreed with this statement. 
The disagreement was somewhat of a consensus with an average 
value of 2.83, a standard deviation of .70 and a median value 
of 3. Disagreement with this statement was widespread 
throughout the DoD acquisition commianity as shown by the group 
data in Table 70. Four of the largest groups of participants, 
contracting officers, DCAA, DCMC, and the U.S. Army had a 
consensus in their disagreement with this issue. 



Table 70 

Question 10b Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Contracting 

Officers 


2.00 


4.00 


2.91 


3.00 


. 61 


3.00 


3.00 


U.S. Army 


2.00 


4.00 


2.93 


3.00 


. 66 


2.25 


3.00 


DCAA 


2.00 


4.00 


2.92 


3.00 


. 66 


2.00 


3.00 


DCMC 


2.00 


4.00 


2.82 


3.00 


. 66 


2.00 


3.00 



Source: Developed by researcher. 
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The depth of feeling over this indicates that CAIV will have 
little effect during negotiations in shifting emphasis to 
other issues. 

3. Minor Impacts on Source Selection 

While cost realism may be a major part -of a source 
selection plan, it should be limited in its impact on other 
areas of an evaluation. All of the proposal component parts 
are interrelated to some degree but the factors should have 
some degree of independence. This statement explores the 
effects of parametric cost estimating methods and cost realism 
in source selection. 

Using parametric cost estimating methods- should have 

little effect on source selection, they only affect cost 

realism. 

The aggregate data for this statement shows disagreement with 
an average response value of 2.66, a standard deviation of 
.78, and a median value of 3. Table 71 contains data from a 
major buying organization, USAF, and two vital components of 
the DoD acquisition community, the auditor and cost estimator 
groups, that disagree with this statement. 

This is contrasted with the regression analysis performed 
with the business size data. The equation and data 

Y = -.360(X) + 7.38 = .18 

(.17) (2.30) 
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display a negative relationship between business size and the 
response value for this statement. As the business size 
increases so does the level of agreement with this statement. 
The industry and corporate policy groups do not show any 
consensus in their opinion on this statement, however. 



Table 71 

Question 10c Group Data 





Min 


Max 


M 


Median 


a 


Q1 


Q3 


Auditors 


1.00 


4.00 


2.56 


2.50 


.70 


2.00 


3.00 


Cost Analysts 


2.00 


4.00 


2.87 


3.00 


. 66 


2.00 


3.00 


USAF 


2.00 


4.00 


3.06 


3.00 


. 68 


3.00 


3.75 



Source: Developed by researcher. 



4 . Significant Impacts on Source Selection 

The interrelated nature of the parts of an offeror' s 

proposal could make the effects of one element have a 

disproportionate effect in the source selection process. 

There was a similar lack of consensus on statement lOd (shown 

below) as there was in statement 10c. 

Using parametric cost estimating methods should have a 
major effect on source selection, they affect past 
performance, technical, management, and cost factors in 
proposal evaluation schemes. 



An average response value of 2.48 and a median value of 2 was 
observed. The distribution of responses was strongly agree. 
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20, agree, 67, disagree, 68, and strongly disagree 16. 
Relatively few people (39) chose no opinion or did not respond 
to this statement. The DCAA. and corporate policy group 
responses in Table 72 are indicative of the range of opinion 
to this statement. They are not, however, indicative of the 
degree of consensus on this issue. These two groups were the 
only ones to show any sort of consensus of the demographic 
groups analyzed. The contrasting opinions between industry 
and DCAA in the acquisition community is clear. The industry 
business size regression analysis results 

Y = +.385(X) - 2.77 = .22 
(.16) (2.21) 

show a positive relationship between the responses and the 
size of the business. The smaller businesses strongly agree 
with the statement while the larger ones merely agree. 



Table 72 

Question lOd Group Data 





Min 


Max 




Median 


a 


Q1 


Q3 


Corporate 

Policy 


1.00 


2.00 


1.75 


2.00 


.50 


1.25 


2.00 


DCAA 


2.00 


4.00 


2.81 


3.00 


.69 


2.00 


3.00 



Source: Developed by researcher. 



The industry responses to this statement and question 10c 
are consistent. The regression analysis demonstrates opposite 
trends for these two contrasting statements, and from this one 
could interpret that the industry portion of the DoD 
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acquisition community views a proposal as a highly integrated 
whole rather than separate sections. The Government, however, 
may not see the issue in the same way. DCAA responding to 
this statement and the technical support group in responding 
to question nine disagree with the idea that using parametric 
cost estimating methods can have significant effects on the 
source selection and subsequent negotiation processes. 

Dennedy, Deutsch, and Hertling also disagree with the 
premise that the use of parametric cost estimating methods can 
have an effect on negotiations. [Ref 8:p.3] The negotiation 
process is based on estimates of all types and parametric 
estimates are only one of many sources of information. In 
total, the results of statements 10a through lOd indicate that 
the DoD acquisition community does not believe that the use of 
parametric cost estimating methods will change the fimdamental 
nature of negotiations. It will simply redefine some of the 
points around which the negotiations may flow. 

F. SUMMARY 

This chapter and the preceding two chapters described the 
data collection efforts and analyses conducted for this 
thesis. Data collection was chiefly done through a ten 
question survey distributed to members of the DoD acquisition 
community. Personal interview information was also collected 
and presented with each question's survey information as it 
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pertained those questions or statements. The combined data 
were examined for indications regarding the overall opinion of 
the entire DoD acquisition community about each issue. The 
demographic group data were then examined and contrasted with 
the entire group responses and between group responses. Any 
groups within the database that had any opinion that was 
markedly different from that of the entire respondent pool 
were highlighted. An analysis of this information was 
presented for each segment of each question. The final 
chapter in this thesis will conclude the study with 
conclusions and recommendations made from the analysis. 
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IX. PRINCIPAL FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 



A. PURPOSE 

The primary purpose of this study was to examine the 
effects of increasing the use of parametric cost estimation 
methods in individual contract actions. This chapter will 
provide conclusions about those effects and recommendations 
for expanding the part that parametric cost estimating methods 
play in Federal Government acquisition processes. Principal 
findings and conclusions will be addressed and then they will 
be followed by the recommendations. Areas of further research 
will be the final section of this thesis. 

B. CONCLUSIONS 

The DoD acquisition community views the use of parametric 
cost estimating as having limited applicability. Areas that 
can be termed traditional uses have the highest degree of 
utility in questions one through four. The acquisition 
community believes that the methods are best applied to 
developmental applications and serve little purpose outside 
those areas. The data support the conclusion that the DoD 
acquisition community is not familiar with the techniques and 
methods used in parametric cost estimating. They do not have 
an appreciation of the benefits that parametric cost 
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estimating can provide across the entire spectrum of activity 
in the DoD acquisition process. 

Practical experience with using the methods in the 
acquisition process and research on the possible applications 
for the methods provide a different view. NASA, industry, and 
the control group responses clearly showed that the methods 
can be used successfully in a far wider range of activities 
than they are normally applied by DoD agencies. They also 
demonstrated areas that have clearly not benefitted by the use 
of parametric cost estimating methods, such as contracting for 
commodities, T&M/Labor hour contract situations, and in most 
areas of the PF/D&OS phase. Parametric cost estimating also 
may be applied in those areas, however, currently they are not 
being used there. The methods may not provide enough 
improvement over existing estimating systems to make 
investments in them cost effective. 

The potential effects of using parametric cost estimating 
methods will require some changes to be made to the way data 
are collected, reviewed, presented, and analyzed. However, 
they will not fundamentally change the acquisition process. 
The DoD acquisition community views parametric cost estimating 
methods as an incremental step in increasing the efficiency of 
the acquisition process. The community acknowledges the fact 
that the methods can affect almost every action in the DoD 
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procurement process. They also note that if the methods are 
to be used, they must be considered from the initial planning 
stages . 

The members of the acquisition community also are viewing 
the use of parametric cost estimating methods • in terms of 
acquisition reform. The data demonstrate a preference for 
giving the contractor community latitude to use the methods as 
they fit best in the contractor' s systems rather than as 
requirements to be met. The data also indicate that the 
acquisition community would accept the idea that teaming 
concepts have potential benefits to parametric cost 
estimating. Major stakeholders in the community identified 
each other as important members of the process whereby a 
parametric cost estimating model is calibrated and validated 
for use. The use of teaming and parametric cost estimating 
fit in well with the concept of disengagement by DoD in its 
contractor estimation system review programs. 

C . RECOMMENDATIONS 

The primary recommendation of this thesis is to maintain 
the Parametric Cost Estimating Initiative and the Joint 
Industry/ Government Working Group. This initiative, through 
the working group, has been instrumental in bringing awareness 
to the benefits that parametric cost estimating can provide. 
The workshops which the working group sponsors and newsletters 
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that it publishes are outstanding fora for publicizing these 
benefits, exploring new areas in which the methods can be 
applied, and producing meaningful discussion about actual 
application of the methods in specific situations. The large 
number of surveys not answered indicates that the majority of 
the DoD acquisition community is still unfamiliar with 
parametric cost estimating methods. 

A second recommendation is to evaluate and publish 
results from the Joint Government/ Industry Parametric Cost 
Estimating Pilot Program. The lessons learned and successes 
will help to promote the use of the methods and also to 
publicize how they can benefit both Government and Industry. 
Business case analyses could be performed with these results 
to quantify the effects of using parametric cost estimating 
methods and implementation procedures. 

Training the acquisition community is the third 
recommendation of this thesis. One of the major efforts of 
the working group has been to establish a training program in 
the area of parametric cost estimating. The training should 
be made readily available to the acquisition community as 
rapidly as possible to reap the maximum benefits from the use 
of the methods. Training should also include a review of the 
existing curricula for acquisition workforce professionals. 
A brief discussion regarding parametric cost estimating 
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methods to introduce them to the workforce could provide basic 
awareness and understanding of the methods if it were added to 
existing cost and pricing course materials. 

Parametric cost estimating methods should be considered 
for incorporation in DoD modeling and simulation programs for 
acquisition purposes. The U.S. Navy Acquisition Center for 
Excellence (ACE) could potentially benefit from the use of the 
methods as well as provide awareness to the acquisition 
community about their potential applications. The U.S. Navy 
Research Development and Acquisition Team Strategic Plan for 
1996 and 1997 provides several other areas where the methods 
could be applied. The strategic goals for total ownership 
cost and innovation/technology insertion discuss modeling and 
simulation areas that could also benefit from using parametric 
cost estimating methods. 

Connectivity between databases should be improved. As 
more of the DoD systems and equipments incorporate COTS and 
NDI, integration will become a major cost driver. Parametric 
estimating could be helpful in modeling the integration 
schedule and costs if databases for the individual equipments 
could be integrated. Parametric estimation of total life 
cycle costs could also benefit from greater connectivity 
between databases. 
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D. FURTHER RESEARCH TOPICS 

The following is a list of several areas that might be of 
interest and benefit to the acquisition community regarding 
parametric cost estimating: 

1. Business case analyses of the Government agencies and 
companies involved in the Joint Government/ Industry 
Parametric Cost Estimating Pilot Program could provide 
greater insights to the application of parametric cost 
estimating methods to DoD acquisition. 

2 . A review of how the use of parametric cost estimating 
methods affect the DoD estimating system review program 
would be useful to buying organizations as well as DoD 
administrative agencies. 

3. Business case analyses of parametric estimating 
applications to the concept of total life cycle costs 
would provide benefits to future programs and DoD 
managers . 

4 . A review of the regulatory and other barriers to 
effective implementation of using parametric costing 
could benefit those interested in promoting the wider use 
of the methods. 



E . FINAL SUMMARY 

The final summary of the issues addressed in this thesis 
will be a restatement of the research questions followed by 
brief answers. 

Primary Research Question: 

How could a DoD emphasis on using parametric methods in 
cost estimating affect the procurement process? 
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Answer: A DoD emphasis on parametric methods in cost 
estimating could influence almost every step in the 
acquisition process in some way. The effects that they 
would cause would be minor in nature, however. 
Sijbsidiary Research Questions: 

1. What changes could offerors make to their proposal 
preparation, submission, and support processes to 
accommodate using parametric methods, techniques, or 
software in estimating potential contract costs? 

Answer: Offerors desiring to incorporate parametric cost 
estimating methods into their proposal preparation 
processes could either use a commercially available model 
or develop one of their own. In either case a database 
from which to work is essential and must be constructed. 
The decisions made here will influence the ability of the 
offeror to respond successfully to Federal Government 
solicitations . 

2. What impacts could emphasis on using cost estimates 
based on parametric estimating techniques have on source 
selection criteria and on the evaluation of proposals by 
DOD? 

Answer: Cost estimates based on parametric estimates 

could impact the entire source selection process. They 
have impacts on the evaluation plan, the proposals 
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submitted, and thus the material available for evaluation 
and source selection decisions. 

3. How could source selection criteria and evaluation 
factors be constructed to fairly evaluate a proposal that 
uses a parametric cost estimate with one that does not? 
Answer: Source selection criteria must be flexible enough 
to include proposals based on parametric estimates as 
well as ones based on traditional estimates. Failing to 
do so could impact the level of competition in a given 
procurement . 

4. What effects could using parametric cost estimations 
have on negotiations? 

Answer: Parametric cost estimating methods should have 
little effect on negotiations. The points around which 
one negotiates may shift to inputs to the model rather 
than specific cost elements but the basics of negotiation 
will remain the same. Cost will always be a factor. 
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APPENDIX A. DEFINITIONS 



Analogy Model - a model that estimates costs of a new program 
or system from data on past costs of similar programs or 
systems. [Ref 23:p.290] 

Analysis of Variance - an expression of the measure of total 
variation in a data set as a sum of terms, which can be 
attributed to specific sources, or causes of variation. [Ref 
37:p.207] 

Benchmarking Process Costs - generating an estimate of the 
costs associated with a process for later comparison with 
other estimates or actual costs. 

Bottoms Up Cost Estimate (Detailed Cost Estimate) - an 
estimate of project component costs from detailed 
understanding of the cost elements of each component. 
Supplier quotations, labor hour estimates from time and motion 
studies or labor hour and material estimates from learning 
curve analyses form the basis of these estimates. [Ref 
2:p.294] 
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Calibration - the process of indexing a parametric model or 
system to local cost and product history. [Ref 5:p.4] 

Capacity Cost Estimates - estimates based on logarithmic plots 
of similar plants which show a relationship between capacity 
and costs. [Ref 14:p.l32] 

Coefficient of Determination - the proportion of variation in 
the dependent variable that has been explained or accounted 
for by the regression line. [Ref 37:p.46] 

Commercial-Of f-The-Shelf - an existing system - or equipment 
that is sold to the Federal Government in the same 
configuration that it is sold to the general public or with 
minor modifications. [Ref 3:p.42] 

Cost Analysis - the review and evaluation of a contractor's 
costs or pricing data, and of the judgmental factors applied 
in projecting from the data to the estimated costs, for the 
purpose of determining the degree to which the contractor's 
proposed costs represent what contract performance should 
cost, assuming reasonable economy and efficiency. [Ref 3:p 19] 
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Cost or Pricing Data - all facts that . . . prudent buyers and 
sellers would reasonably expect to affect price negotiations 
significantly. [Ref 13:part 15.801] 

Cost Realism - means the costs in an of feror ' s- proposal are 
realistic for the work to be performed, reflect a clear 
understanding of the requirements and are consistent with the 
various elements of the offeror's technical proposed. [Ref 
13:part 15.801] 

Dependent Variable - the variable whose value is to be 
predicted. [Ref 37:p.39] 

Equipment-Ratio Estimates - estimates derived from process 
flow diagrams and equipment lists with installation and 
construction costs estimated as a ratio of past similar 
efforts. [Ref 14;p.l32] 

Expert Opinion - the use of the judgment of experts when 
supporting data or program definitions are insufficient. [Ref 
23:p.290] 

Independent Variable - the variable about which knowledge can 
be obtained. [Ref 37:p.39] 
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Layout Estimates - estimates from plant layouts and process 
flows and equipment lists. [Ref 14:p.l32] 

Learning Curve - an empirical relationship between the number 
of units of an item produced and the number of units of a 
resource used to produce them. The most common relationship 
used is units of production and nxamber of labor hours required 
to produce the units. [Ref l:p.267] 

Negotiation - a process between buyers and sellers seeking to 
reach mutual agreement on a matter of common concern through 
fact finding, bargaining, and persuasion. [Ref 3:p.41] 

Non-developmental Item - an item developed exclusively at 
private expense and sold in substantial quantities, on a 
competitive basis, to multiple state and local governments. 
[Ref 13:part 2.101] 

Parametric Cost Estimate - an estimate derived from 
statistical correlation of historic system costs with 
performance and/or physical attributes of the system, [Ref 
37:p.l72] 
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Performance Specification - a description of a deliverable in 
terms of desired operational capabilities. [Ref 3:p.44] 

Preliminary Bill Estimates - estimates derived from bills of 
material produced from initial drawings and labor-material 
ratios from past work. [Ref 14:p.l32] 

Price Analysis - the process of examining and evaluating a 
proposed price without evaluating its separate cost elements 
and proposed profit. [Ref 13:part 15.8] 

Regression Analysis - the mathematical nature of the 
association between two variables. [Ref 37:p.39] 

Risk - the probability of not attaining the goals for which 
the party entered a contract. [Ref 3:p.51] 

Source Selection - the process wherein the requirements, 
facts, recommendations and policies relevant to an award 
decision in a competitive procurement of a system/project are 
examined and the decision is made. [Ref 3:p.53] 
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standard Error of the Estimate - the standard deviation of the 



population of y values predicted from a single value of x. 
[Ref 23:p.305] 

Top Down Cost Estimate - an overall cost estimate for a 
proposed project based on the global properties of the project 
from which component parts are partitioned for planning 
purposes. [Ref 34:p.l76] 

Validation - the process or act of demonstrating the 
calibrated model's ability to function as a credible forward 
estimating tool or system. [Ref 5:p.4] 

Work Breakdown Structure - a product-oriented family tree 
resulting from system engineering efforts which completely 
define the program. [Ref 37:p.l76] 
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APPENDIX B. LIST OF ACRONYMS 



Sd^ - Sum of the Squares 
- Population Variance 
ACE - Acquisition Center of Excellence 
ACO - Administrative Contracting Officer 
ACT - Active Coefficient 
CAIG - Cost Analysis Improvement Group 
CAIV - Cost As An Independent Variable 
CAS - Cost Accounting Standards 
CE - Concept Exploration 
CER - Cost Estimating Relationship 
CICA - Competition in Contracting Act 
COCOMO - Constructive Cost Model 
COTS - Commercial off the Shelf 
CPAF - Cost-Plus-Award-Fee Contract 
CPFF - Cost-Plus-Fixed-Fee Contract 
CPIF - Cost-Plus-Incentive-Fee Contract 
CPRG - Contract Pricing Reference Guides 
DCAA - Defense Contract Audit Agency 
DCMC - Defense Contract Management Command 
DEVC - Development Cost Variable 
DF - Degrees of Freedom 

DFARS - Defense Federal Acquisition Regulation Supplement 
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DLSIE - Defense Logistics Studies Information Exchange 
DoD - Department of Defense 
DoE - Department of Energy 
DoN - Department of Navy 

DTIC - Defense Technical Information Center 
ECP - Engineering Change Proposal 
E&MD - Engineering and Manufacturing Development 
F - F Statistic 

FAR - Federal Acquisition Regulation 
FFP - Firm Fixed-Price Contract 

FPE - Fixed-Price with Economic Price Adjustment Contract 
FPIF - Fixed-Price Incentive Firm Contract 
FPIS - Fixed-Price Incentive, Successive Targets Contract 
FPRA - Forward Pricing Rate Agreement 

FPR(P) - Fixed-Price with Prospective Redetermination Contract 
FPR(R) - Fixed-Ceiling-Price with Retroactive Redetermination 
Contract 

ICE - Independent Cost Estimate 
GS - General Service 

GSA - General Services Administration 

IPCE - Independent Parametric Cost Estimate 

LG - Large Ship Factor 

LRIP - Low Rate Initial Production 

LSBF - Least Squares Best Fit 



208 



NASA - National Aeronautics and Space Administration 

NDI - Non-developmental Item 

OSD - Office of the Secretary of Defense 

P - P value 

PCO - Procuring Contracting Officer 

PD&RR - Program Definition and Risk Reduction 

PF/DOS - Production, Fielding/Deployment and Operational Support 
PRICE - Programmed Review of Information for Costing and 
Evaluation 

PROC - Procurement Cost Variable 
R^ - Coefficient of Determination 

RDT&E - Research, Development, Test, and Evaluation 

SES - Senior Executive Service 

SM - Small Ship Factor 

SOW - Statement of Work 

SPI - Single Process Initiative 

SSE - Standard Sum of the Errors 

T&M - Time and Materials Contract 

TINA - Truth in Negotiations Act 

USAF - United States Air Force 

USA - United States Army 

USN - United States Navy 

WBS - Work Breakdown Structure 
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APPENDIX C. SURVEY QUESTIONS 



Please use the following as a definition, taken from the DoD 
Parametric Cost Estimating Handbook published in the fall of 
1995, for Parametric Cost Estimating (PCE) : 

"A technique employing one or more Cost Estimating 
Relationships (CERs) and associated mathematical relationships 
and logic. The technique is used to measure and/or estimate 
the cost associated with the development, manufacture, or 
modification of a specified end item. The measurement is 
based on the technical, physical, or other end item 
characteristics . " 

PARAMETRIC COST ESTIMATING SURVEY 

Please complete the following demographic information. 

1. What is your job classification? (Select 1 that fits best or write in a specific title if none apply.) 

Cost Estimator Cost/Price Analyst Contract Negotiator 

Contracting Officer Program Staff Program Manager Engineer 

Legal Support Technical Support Auditing Accounting 

Other ^ 

2. If employed by industry, annually does your company do business with the Federal 
Government totaling; 

<$100 Million $ 1 00 - 500 Million $ 500 - $ 1 BUlion > $ 1 BilUon 



3. If employed by the Federal Government: 

^USA DCMC DCAA ^NASA USN USAF GSA 

Other: GS/SES Level/Rank 

4. How long have you been working in the cost estimating or acquisition fields? 

(Please select either career field and the appropriate length of time.) 

One year or less 1-5 years 5-10 years 10-20 years Over 20 years 

5. What functions involving cost estimating do you primarily perform? 

^Prepare Estimates ^Use in Negotiation Preparation or Proposal Analysis 

R eview/Audit Estimates Set Corporate or ^Federal Government Policy 

6. What is your level of mathematics knowledge? 

High School Algebra/Calculus College Algebra College Business Calculus 

College Engineering Calculus Series College Mathematics Minor/Major 

Advanced Degree Involving Training in Mathematics Other 
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Please record your opinions regarding the following questions relating to PCE. 

For questions 1 through 4, use the following scale to represent the utility of PCE in the listed 
areas. 

(1) Vital (2) Very Useful (3) Somewhat Useful (4) No Use (5) Hinders the Process 

1 . How useful is PCE in these areas? 

Bench Marking Process Costs Validating Bottom’s Up Estimates 

Program Estimates ^Budgeting ICE Generation Software Contracts 

^Repair Parts Contracts R&D Contracts Service Contracts 

^Hardware Contracts Construction Contracts Commodity Contracts 

PCE is also useful for: (Please fill in other areas.) 

2. How useful is PCE in each of the stages of a system’s life cycle? 

Concept Program Definition Engineering & Manufacturing Production, Fielding/ 
Exploration & Risk Reduction Development Deployment & Operational 

Support 



3. How useful is PCE in situations where you would also use the following contract type: 
FFP FPE FPIS FPR(P) FPR(R) FPIF CPIF CPAF CPFF T&M Labor Hour 



PCE is also useful for this contract type situation . (Please fill in others.) 

4. How useful is PCE in estimating costs for activities in these two- areas of contract 
management? 

Pre-Award Actions Contract Post-Award Actions 

This part is a series of statements about the use of PCE methods. Using the following scale, 
please indicate to the left of each statement, the degree to which you agree or disagree with 
them. 

(1) Strongly Agree (2) Agree (3) Disagree (4) Strongly Disagree (5) No Opinion 

5. PCE methods, models and software: 

a. Could eventually replace traditional “bottoms up” cost estimating methods in many 

acquisition estimating applications as a result of acquisition streamlining. 

b. Could replace traditional cost estimating methods as larger, more compatible data bases, 

greater computing power and improved software become available. 

c. Can save contractors and Federal Government oversight officials significant time and 

expense in developing and then evaluating proposals. 

d. Could significantly change the way potential contractors collect and summarize cost 

information. 

e. Should decrease the probability of cost overruns in programs where used. 

f Should change the way Cost and Pricing Data submitted to support a proposal is viewed. 

6. In solicitations, buying activities should: 

a. Encourage that PCE be used for specified procurement actions. 

b. Specify PCE methods, models, or software for cost proposal construction. 

c. Leave PCE methods, models, or software selection to contractor discretion. 

d. Require a specific cost breakdown format for PCE based cost proposals. 
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e. Require a specific format to extract labor hour, subcontractor, or material quantity 

estimates for technical proposal evaluation. 

f State criteria to be used to evaluate the statistical significance of CERs used to support 

a PCE model. 

g. Require cost proposals based on either PCE methods or traditional costing methods. 

Evaluation schemes cannot fairly distinguish between proposals using different costing 
methods for the same work. 

7. A potential contractor should provide the foDowing to support a cost proposal: 

a. PCE model software and input data used. 

b. Justification to support the use of PCE models and input data. 

c. Data and rationale used to validate a PCE model for a specific set of circumstances. 

8. DCAA should perform the following activities: (If not DCAA, who should do them?) 

a. Establish metrics to determine a PCE model’s validity in a given situation. 

b. Establish criteria to determine when a PCE model should be updated. 

c. Verify that a PCE model accurately predicts costs for the expected contract work. 

d. “Run” a contractor’s PCE model for purposes of analyzing the cost proposal. 

9. Construction of CERs used in PCE based proposals should be used as a factor in 

evaluating a potential contractor’s level of understanding of the technical requirements in a 
solicitation. 

10. Using PCE methods: 

a. Should make negotiating costs less critical as cost estimate accuracy improves. 

b. Should have little effect on negotiations as the concept of Cost as an Independent 

Variable shifts emphasis to technical and schedule concerns for a given value. 

c. Should have little affect on source selection, they only affect a proposal’s cost realism. 

d. Should have a major affect on source selection, they affect past performance, technical, 

management, and cost factors in proposal evaluation schemes. 
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APPENDIX D. INTERVIEW QUESTIONNAIRE 



1. How should an emphasis on using parametric methods in cost 

estimating affect the procurement process? 

2. How could the use of parametric cost estimates be incorporated 

into program management beyond their traditional uses? 

3. How far should DoD/DoN go in emphasizing parametric cost 
estimating methods? 

4 . Should the use of parametric cost estimates be required for 
certain procurement actions? 

5. What information should DoD/DoN agencies expect potential 

contractors to provide in support of their proposals that use 
parametric cost estimating techniques for proposal 
preparation? 

6. Do you see any major changes to the way solicitations are 
structured, proposals are generated, proposals evaluated, and 
sources selected because of an emphasis on using parametric 
cost estimating techniques? 

7. What support should DoD/DoN organizations expect of DCAA and 
DCMC for analyzing parametric cost estimate based proposals? 

8 . Should DoD/DoN provide guidance on how the variance between a 
parametric cost estimating model and the work described on a 
solicitation be measured? 
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APPENDIX E. COLLECTED DATA 



DEMOGRAPHIC GROUP INFORMATION 



Group Name 


Sort 

Code 


#of 

Responses/ 

%of 

Responses 


Math 

Sort 

Code 

IJ- 


Math 

Sort 

Code 

o 


Experience 
Sort Code 

U- 


Experience 
Sort Code 
a 


Control Group 




11 


5.3% 










Job Classification 














Cost Estimator 


1 


14 


6.7% 


38.5 


1.3 


28.1 


.89 


Cost/Price Analyst 


2 


55 


26.2% 


37.6 


1.43 


28.3 


.68 


Contract Negotiator 


3 


8 


3.8% 


36.9 


1.46 


27.5 


1.00 


Contracting Officer 


4 


49 


23.3% 


37.1 


1.47 


28.5 


.56 


Program Staff 


5 


5 


2.4% 


37.8 


1.47 


28.6 


.49 


Program Manager 


6 


7 


3.3% 


38.4 


1.18 


27.9 


.99 


Engineer 


7 


12 


5.7% 


38.8 


1.09 


28.1 


.90 


Legal Support 


8 


0 










Technical Support 


9 


5 


2.4% 


38.6 


1.20 


28.2 


.75 


Auditor 


10 


46 


21.2% 


36.5 


.95 


28.6 


.53 


Accounting 


11 


2 


1% 


38.0 


2.00 


27.5 


.5 


Other 




7 


3.3% 










Industry Size 














<$100 Million 


12 


2 


1% 


38.0 


1.00 


28.0 




$100 - $500 Million 


13 


8 


3.8% 


38.6 


1.26 


28.2 


.79 


$500 Million - $1 Billion 


14 


1 


.5% 


39.0 








> $1 Billion 


15 


9 


4.3% 


39.0 


1.00 


28.3 


.66 


Total Industry 




20 


10% 
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Group Name 


Sort 

Code 


# of 

Responses 


Math 

Sort 

Code 


Math 

Sort 

Code 

o 


Experience 
Sort Code 

At 


Experience 
Sort Code 
o 


Government 














U.S. Army 


16 


52 


24.9% 


37.7 


1.59 


28.4 


.72 


DCMC 


17 


35 


16.8% 


37.1 


1.36 


26.1 


.73 


DCAA 


18 


49 


22.9% 


36.6 


1.06 


28.6 


.58 


NASA 


19 


9 


4.4% 


38.0 


1.41 


28.0 


1.00 


U.S. Navy 


20 


22 


10.6% 


37.3 


1.33 


28.2 


.89 


USAF 


21 


17 


8.2% 


37.9 


1.49 


28.2 


.76 


GSA 


22 


1 


.6% 


37.0 




28.0 




OSD 




2 


1% 










DoE 




2 


1% 










Total Government 




189 


90% 










Level/Rank 














GS 12/03 




7 


3.7% 










GS 13/04 




19 


9.1% 










GS 14/05 




24 


11.4% 










GS 15/06 




25 


11.5% 










Experience 














Cost Estimating 
Community 


23 


40 


19.1% 


37.2 


1.43 


28.0 


.81 


Acquisition Community 


24 


102 


48.6% 


38.0 


1.54 


28.5 


.50 


One Year or Less 


25 


0 










One to Five Years 


26 


5 


2.3% 


37.4 


.80 






Five to Ten Years 


27 


19 


9.1% 


37.8 


1.54 






Ten to Twenty Years 


28 


85 


40.5% 


37.6 


1.57 
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Group Name 


Sort 

Code 


#of 

Responses 


Math 

Sort 

Code 

1^ 


Math 

Sort 

Code 

a 


Experience 
Sort Code 


Experience 
Sort Code 
a 


Over Twenty Years 


29 


97 


46.2% 


37.2 


1.41 






Work Description 














Prepare Estimates 


30 


37 


17.6% 


38.1 


.70 


28.1 


.91 


Negotiation Preparation 


31 


101 


48.1% 


37.2 


1.46 


28.3 


.73 


Review/ Audit Estimates 


32 


103 


49.1% 


37.4 


1.52 


28.4 


.65 


Set Corporate Policy 


33 


2.4% 


39.0 


1.26 


28.0 


.63 


Set Government Policy 


34 


28 


13.3% 


38.2 


1.54 


28.3 


.76 


Mathematics Knowledge 














High School 
Algebra/Calculus 


35 


24 


11.4% 






28.3 


.55 


College Algebra 


36 


64 


30.5% 






28.5 


.58 


College Business Calculus 


37 


69 


32.9% 






28.2 


.80 


College Engineering 
Calculus 


38 


34 


16.2% 






28.2 


.82 


College Mathematics 
Major or Minor 


39 


31 


14.8% 






28.6 


.67 


Advanced Degree With 
Mathematics 


40 


28 


13.3% 






28.1 


.65 



SURVEY DISTRffiUTION AND RETURN DATA 



Group 

Name 


# Surveys 
Sent 


# Incorrect 
Addresses 


# Surveys 
Returned 


%of 

Surveys 

Sent 


Group 

Response 

Rate 


% of Surveys 
Returned 


U.S. Army 


332 


4 


52 


26.1 


15.9 


24.9 


DCMC 


203 


0 


35 


16.1 


17.2 


16.8 
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Group 

Name 


# Surveys 
Sent 


# Incorrect 
Addresses 


# Surveys 
Returned 


%of 

Surveys 

Sent 


Group 

Response 

Rate 


% of Surveys 
Returned 


DCAA 


525 


131 


49 


31.2 


12.2 


22.9 


NASA 


39 


0 


9 


3.0 


23.1 


4.4 


U.S. Navy 


120 


0 


22 


9.5 


18.3 . 


10.6 


USAF 


80 


10 


17 


5.6 


24.3 


8.2 


GSA 


9 


0 


1 


.7 


11.1 


.6 


DoE 


55 


0 


2 


4.4 


3.6 


1 


OSD 


11 


0 


2 


.9 


18.2 


1 


Industry 


33 


0 


20 


2.5 


64.5 


9.6 


Total 


1407 


145 


209 


100 


Avg 16.5 


100 
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TOTAL GROUP DATA 



Responses 




No 

Ans 


#1 


2 


3 


4 


5 




Avg 


Std 

Dev 


Var 


Med 


Skew 


QUESTION 1; 


























Bench Maricmg Process Costs 


76 


14 


42 


56 


21 


1 




2.66 


.89 


.75 


3.00 


-0.14 


Validating Bottom^s Up Estimates 


50 


17 


52 


71 


17 


3 




2.61 


.88 


.78 


3.00 


-0.22 


Program Estimates 


54 


29 


64 


56 


7 


0 




2.25 


.82 


.81 


2.00 


-0.21 


Budgeting 


48 


37 


68 


48 


9 


0 




2.20 


.85 


.60 


2.00 


-0.35 


ICE Generation 


90 


25 


41 


39 


15 


0 




2.34 


.95 


.48 


2.00 


-0.37 


Software extracts 


71 


27 


29 


54 


27 


2 




2.54 


1.06 


.84 


3.00 


-0.59 


Repair Parts 


66 


8 


34 


72 


30 


0 




2.85 


.80 


1.27 


3.00 


0.00 


R&D 


57 


18 


41 


49 


40 


5 




2.83 


1.06 


.76 


3.00 


-0.56 


Service 


58 


8 


29 


57 


57 


1 




3.09 


.89 


77 


3.00 


-0.79 


Hardware 


47 


29 


56 


59 


19 


0 




2.55 


.85 


.85 


3.00 


-0.64 


Construction 


79 


11 


41 


54 


25 


0 




2.71 


.88 


.48 


3.00 


-0.65 


Commodity 


101 


3 


13 


51 


41 


1 




3.23 


.78 


.64 . 


3.00 


0.65 


QUESTION!: 


























CE 


45 


39 


48 


43 


33 


2 




2.46 


1.09 


1.27 


2.00 


0.89 


PDR&R 


40 


30 


60 


58 


21 


1 




2.43 


.95 


.80 


2.00 


0.43 


E&MD 


40 


22 


62 


76 


10 


0 




2.45 


.82 


.41 


3.00 


-0.59 


PFD&OS 


42 


19 


52 


70 


27 


0 




2.65 


.89 


.48 


3.00 


-0.05 


QUESTION 3: 


























FFP 


52 


19 


55 


60 


23 


1 




2.56 


.90 


1.22 


3.00 


0.60 


FPE 


81 


12 


38 


53 


25 


1 




2.71 


.90 


1.22 


3.00 


0.60 


FPIS 


83 


11 


42 


59 


15 


0 




2.62 


.81 


1.04 


3.00 


1.02 


FPR(P) 


77 


12 


46 


60 


15 


0 




2.60 


.81 


1.04 


3.00 


1.02 


FPR(R) 


79 


13 


42 


57 


19 


0 




2.64 


.85 


1.04 


3.00 


1.02 


FPIF 


69 


14 


52 


57 


18 


0 




2.56 


.83 


1.22 


3.00 


0.60 


CPIF 


65 


19 


43 


59 


24 


0 




2.62 


.91 


.92 


3.00 


0.54 


CPAF 


68 


20 


43 


55 


23 


1 




2.59 


.95 


.92 


3.00 


0.54 


CPFF 


65 


22 


41 


59 


22 


1 




2.58 


.95 


.92 


3.00 


0.54 


T&M 


69 


7 


26 


53 


49 


6 




3.13 


97 


.50 


3.00 


0.94 


LABOR HOUR 


73 


5 


30 


47 


49 


6 




3.16 


.93 


.56 


3.00 


0.62 
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No 

Ans 


#1 


2 


3 


4 


5 




Avg 


Std 

Dev 


Var 


Med 


Skew 


QUESTION 4 : 


























PRE-AWARD 


32 


32 


64 


69 


13 


0 




2.63 


.86 


.60 


2.00 


- 0.35 


POST-AWARD 


41 


17 


32 


59 


53 


8 




3.04 


1.05 


1.26 


3.00 


- 0.50 


QUESTION 5: 


























A 


0 


28 


62 


67 


26 


27 




Z 49 


.92 


.61 


3.00 


- 0.18 


B 


0 


20 


82 


56 


15 


37 




2.37 


.81 


.43 


2.00 


- 1.32 


C 


0 


45 


97 


37 


8 


23 




2.04 


.79 


.55 


2.00 


0.00 


D 


0 


34 


103 


35 


11 


27 




2.15 


.83 


1.17 


2.00 


0.79 


E 


0 


7 


43 


71 


39 


50 




2.89 


.82 


.54 


3.00 


- 0.81 


F 


0 


18 


99 


41 


13 


39 




2.29 


.76 


.44 


2.00 


- 0.43 


QUESTION 6: 


























A 


0 


27 


95 


40 


12 


36 




2.21 


.78 


.69 


2.00 


0.69 


B 


0 


14 


49 


84 


29 


34 




2.73 


.83 


.74 


3.00 


- 0.34 


C 


0 


22 


89 


40 


18 


41 




2.33 


.84 


.56 


2.00 


0.00 


D 


0 


24 


84 


55 


13 


34 




2.35 


.80 


1.29 


200 


0.23 


E 


0 


26 


77 


56 


15 


36 




2.37 


.83 


1.24 


2.00 


0.50 


F 


0 


36 


101 


28 


9 


36 




2.06 


.76 


.32 


2.00 


- 0.77 


G 


0 


13 


66 


60 


20 


51 




2.55 


.80 


.89 


3.00 


- 0.89 


QUESTION 7: 


























A 


0 


66 


95 


17 


7 


25 




1.81 


.76 


.51 


2.00 


0.33 


B 


0 


64 


107 


13 


3 


23 




1.75 


.65 


.23 


2.00 


- 0.66 


C 


0 


71 


106 


6 


3 


24 




1.67 


.61 


.23 


2.00 


- 0.66 


QUESTION 8: 


























A 


0 


31 


90 


35 


19 


35 




225 


.87 


.69 


2.00 


- 0.36 


B 


0 


32 


93 


33 


16 


36 




2.20 


.84 


.94 


2.00 


- 0.20 


C 


0 


50 


102 


19 


8 


31 




1.93 


.75 


1.23 


2.00 


0.35 


D 


0 


30 


99 


24 


9 


48 




208 


.74 


.94 


2.00 


0.23 


QUESTION 9: 


0 


15 


82 


37 


13 


63 




2.37 


.83 


.03 


2.00 


- 0.58 


QUESTION 10: 


























A 


0 


10 


80 


66 


22 


32 




2.57 


.80 


.69 


2.00 


- 0.41 


B 


0 


1 


48 


84 


20 


57 




2.83 


.70 


.98 


3.00 


0.51 


C 


0 


9 


66 


78 


21 


36 




2.66 


.78 


.56 


3.00 


0.57 


D 


0 


20 


67 


68 


16 


39 




2.48 


.85 


.65 


2.00 


- 0.65 
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Control Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 




Std Dev 


Variation 


Median 


Skew 


Question 1 
























Program Est 


0 


8 


1 


2 


0 


0 


1.45 


0.78 


0.61 


1 


1.50 


Budgeting 


0 


5 


4 


1 


1 


0 


1.82 


0.94 


0.88 


2 


1.20 


R&D 


1 


6 


2 


2 


0 


0 


1.60 


0.80 


0.64 


1 


0.61 


Software 


2 


0 


5 


2 


1 


1 


2.78 


1.03 


1.06 


2 


0.10 


Repair Parts 


1 


4 


2 


4 


0 


0 


2.00 


0.89 


0.80 


2 


(0.15) 


Service 


2 


1 


2 


1 


3 


2 


3.33 


1.33 


1.78 


4 


(0.33) 


Hardware 


2 


2 


5 


2 


0 


0 


2.00 


0.67 


0.44 


2 


(0.45) 


Construction 


3 


2 


2 


3 


1 


0 


2.38 


0.99 


0.98 


3 


0.07 


Commodity 


5 


0 


0 


2 


1 


3 


4.17 


0.90 


0.81 


5 


0.08 


















































Question 2 
























CE 


0 


7 


3 


1 


0 


0 


1.45 


0.66 


0.43 


1 


1.32 


PD&RR 


0 


5 


6 


0 


0 


0 


1.55 


0.50 


0.25 


2 


(0.21) 


E&MD 


0 


4 


4 


3 


0 


0 


1.91 


0.79 


0.63 


2 


0.19 


PF/D&OS 


0 


4 


1 


2 


4 


0 


2.55 


1.30 


1.70 


3 


(0.12) 


















































Question 3 
























FFP 


2 


4 


3 


1 


0 


1 


2.00 


1.25 


1.56 


2 


1.28 


tpE 


4 


3 


2 


1 


0 


1 


2.14 


1.36 


1.84 


2 


1.36 


FPIS 


5 


3 


1 


2 


0 


0 


1.83 


0.90 


0.81 


2 


0.89 


FPR(P) 


5 


3 


1 


2 


0 


0 


1.83 


0.90 


0.81 


2 


0.89 


FPR(R) 


5 


3 


2 


1 


0 


0 


1.67 


0.75 


0.56 


2 


0.86 


FPIF 


3 


4 


3 


1 


0 


0 


1.63 


0.70 


0.48 


2 


0.35 


CPIF 


3 


4 


4 


0 


0 


0 


1.50 


0.50 


0.25 


2 


(0.19) 


CPAF 


3 


4 


4 


0 


0 


0 


1.50 


0.50 


0.25 


2 


(0.19) 


CPFF 


3 


5 


3 


0 


0 


0 


1.38 


0.48 


0.23 


1 


0.00 


T&M 


4 


1 


1 


2 


1 


2 


3.29 


1.39 


1.92 


3 


0.29 


Labor Hour 


5 


1 


1 


1 


0 


3 


3.50 


1.61 


2.58 


4 


0.61 


























Question 4 
























Pre-Award 


2 


9 


0 


0 


0 


0 


1.00 


0.00 


0.00 


1 


(1.92) 


Post-Award 


3 


7 


1 


0 


0 


0 


1.13 


0.33 


0.11 


1 


0.03 



223 





No Response 


#1 


#2 


#3 


#4 


#5 




Std Dev 


Variation 


Median 


Skew 


Question 5 
























5a 


0 


7 


2 


2 


0 


0 


1.55 


0.78 


0.61 


1 


1.15 


5b 


11 


0 


0 


0 


0 


0 


0.00 


0.00 


0.00 


0 


0.00 


5c 


0 


8 


3 


0 


0 


0 


1.27 


0.45 


0.20 


1 


1.19 


5d 


0 


3 


5 


1 


0 


2 


2.36 


1.37 


1.87 


2 


1.21 


5e 


11 


0 


0 


0 


0 


0 


0.00 


0.00 


0.00 


0 


0.00 


5f 


11 


0 


0 


0 


0 


0 


0.00 


0.00 


0.00 


0 


0.00 


























Question 6 
























6a 


0 


0 


4 


4 


3 


0 


2.91 


0.79 


0.63 


3 


0.19 


6b 


0 


0 


1 


7 


3 


0 


3.18 


0.57 


0.33 


3 


(0.03) 


6c 


11 


0 


0 


0 


0 


0 


0.00 


0.00 


0.00 


0 


0.00 


6d 


1 


1 


1 


6 


2 


0 


2.90 


0.83 


0.69 


3 


(1.23) 


6e 


1 


1 


3 


4 


2 


0 


2.70 


0.90 


0.81 


3 


(0.69) 


6f 


0 


1 


6 


3 


1 


0 


2.36 


0.77 


0.60 


2 


0.54 


6g 


11 


0 


0 


0 


0 


0 


0.00 


0.00 


0.00 


0 


0.00 


























Question 7 
























7a 


0 


3 


5 


1 


2 


0 


2.18 


1.03 


1.06 


2 


0.74 


7b 


0 


4 


6 


1 


0 


0 


1.73 


0.62 


0.38 


2 


0.29 


7c 


0 


4 


6 


1 


0 


0 


1.73 


0.62 


0.38 


2 


0.29 


























Question 8 
























8a 


0 


0 


5 


4 


2 


0 


2.73 


0.75 


0.56 


3 


0.57 


8b 


0 


2 


7 


2 


0 


0 


2.00 


0.60 


0.36 


2 


0.00 


8c 


0 


4 


2 


5 


0 


0 


2.09 


0.90 


0.81 


2 


(0.21) 


8d 


11 


0 


0 


0 


0 


0 


0.00 


0.00 


0.00 


0 


0.00 


























Question 9 
























9 


0 


1 


4 


3 


2 


1 


2.82 


1.11 


1.24 


3 


0.42 


























Question 10 
























10a 


0 


1 


5 


2 


1 


2 


2.82 


1.27 


1.60 


2 


0.71 


10b 


11 


0 


0 


0 


0 


0 


0.00 


0.00 


0.00 


0 


0.00 


10c 


1 


2 


6 


2 


0 


0 


2.00 


0.63 


0.40 


2 


(0.69) 


lOd 


1 


0 


2 


5 


3 


0 


3.10 


0.70 


0.49 


3 


(1.42) 
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Cost Estimator Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Oev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


4 


2 


3 


3 


2 


0 


2.50 


1.02 


1.05 


3 


0.09 


Validating 


0 


1 


6 


5 


1 


1 


2.64 


0.97 


0.94 


3 


0.86 


Program Est 


2 


3 


4 


4 


1 


0 


2.25 


0.92 


0.85 


2 


(0.15) 


Budgeting 


1 


4 


5 


4 


0 


0 


2.00 


0.78 


0.62 


2 


(0.31) 


ICE 


3 


1 


4 


5 


1 


0 


2.55 


0.78 


0.61 


3 


(0.49) 


Software 


3 


2 


2 


4 


2 


1 


2.82 


1.19 


1.42 


3 


(0.03) 


Repair Parts 


5 


1 


1 


1 


6 


0 


3.33 


1.05 


1.11 


4 


(0.16) 


R&D 


2 


3 


2 


6 


1 


0 


2.42 


0.95 


0.91 


3 


(0.42) 


Service 


3 


1 


1 


5 


4 


0 


3.09 


0.90 


0.81 


3 


(0.70) 


Hardware 


2 


3 


5 


2 


0 


0 


1.90 


0.95 


0.91 


3 


(0.46) 


Construction 


3 


1 


1 


8 


1 


0 


2.82 


0.72 


0.51 


3 


(0.87) 


Commodity 


7 


1 


1 


4 


1 


0 


2.71 


0.88 


0.78 


3 


0.46 


















































Question 2 
























CE 


0 


6 


5 


1 


2 


0 


1.93 


1.03 


1.07 


2 


1.04 


PD&RR 


1 


3 


6 


2 


2 


0 


2.23 


0.97 


0.95 


2 


0.20 


E&MD 


1 


1 


7 


5 


0 


0 


2.31 


0.61 


0.37 


2 


(1.14) 


PFD&OS 


1 


0 


3 


7 


3 


0 


3.00 


0.63 


0.40 


3 


(1.52) 


















































Question 3 
























=FP 


7 


1 


2 


1 


3 


0 


2.86 


1.12 


1.27 


3 


0.64 


FPE 


7 


1 


2 


1 


3 


0 


2.86 


1.12 


1.27 


3 


0.64 


FPIS 


8 


1 


2 


1 


2 


0 


2.67 


1.11 


1.22 


3 


1.00 


FPR(P) 


J 


1 


2 


1 


2 


0 


2.67 


1.11 


1.22 


3 


i.ooi 


FPR(R) 


8 


1 


2 


1 


2 


0 


2.67 


1.11 


1.22 


3 


1.00 


FPIF 


7 


1 


2 


1 


3 


0 


2.86 


1.12 


1.27 


3 


0.64 


CPIF 


7 


1 


2 


2 


2 


0 


2.71 


1.03 


1.06 


3 


0.63 


CPAF 


7 


1 


2 


2 


2 


0 


2.71 


1.03 


1.06 


3 


0.63 


CPFF 


7 


1 


2 


2 


2 


0 


2.71 


1.03 


1.06 


3 


0.63 


T&M 


8 


0 


1 


2 


3 


0 


3.33 


0.75 


0.56 


4 


0.56 


Labor Hour 


7 


0 


2 


2 


3 


0 


3.14 


0.83 


0.69 


3 


0.37 


























Question 4 
























Pre-Award 


1 


4 


4 


5 


0 


0 


2.08 


0.83 


0.69 


2 


(0.38) 


Post-Award 


1 


2 


2 


3 


5 


1 


3.08 


1.18 


1.40 


3 


(0.56) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


0 


1 


5 


7 


1 


0 


2.57 


0.73 


0.53 


3 


(0.28) 


5b 


0 


1 


5 


8 


0 


0 


2.50 


0.63 


0.39 


3 


(0.98) 


5c 


0 


4 


5 


5 


0 


0 


2.07 


0.80 


0.64 


2 


(0.14) 


5d 


0 


4 


6 


2 


2 


0 


2.14 


0.99 


0.98 


2 


0.67 


5e 


2 


0 


3 


5 


4 


0 


3.08 


0.76 


0.58 


3 


(1.03) 


5f 


1 


0 


8 


3 


2 


0 


2.54 


0.75 


0.56 


2 


(0.33) 


Question 6 
























6a 


1 


2 


6 


4 


1 


0 


2.31 


0.82 


0.67 


2 


(0.32) 


6b 


1 


1 


3 


7 


2 


0 


2.77 


0.80 


0.64 


3 


(1.05) 


6c 


2 


1 


9 


1 


1 


0 


2.17 


0.69 


0.47 


2 


(0.17) 


6d 


0 


4 


3 


4 


3 


0 


2.43 


1.12 


1.24 


3 


0.03 


6e 


4 


6 


2 


2 


0 




1.60 


0.99 


0.98 


2 


0.67 


6f 


4 


1 


6 


3 


0 


0 


2.20 


0.60 


0.36 


2 


(0.38) 


6g 


1 


1 


3 


6 


3 


0 


2.85 


0.86 


0.75 


3 


(0.94) 


Question 7 
























7a 


2 


4 


6 


2 


0 


0 


1.83 


0.69 


0.47 


2 


(0.24) 


7b 


2 


4 


8 


0 


0 


C 


1.67 


0.47 


0.22 


2 


(0.97^ 


7c 


2 


4 


8 


0 


0 


0 


1.67 


0.47 


0.22 


2 


(0.97^ 


Question 8 
























8a 


3' 


1 


5 


3 


2 


0 


2.55 


0.89 


0.79 


2 


(0.21) 


8b 


4 


1 


3 


3 


3 


0 


2.80 


0.98 


0.96 


3 


(0.14) 


8c 


4 


2 


4 


1 


3 


0 


2.50 


1.12 


1.25 


2 


0.27 


8d 


5 


2 


4 


2 


1 


0 


2.22 


0.92 


0.84 


2 


0.40 


























Question 9 
























9 


5 


0 


7 


1 


1 


0 


2.33 


0.63 


0.40 


2 


0.13 


Question 10 
























10a 


0 


0 


4 


4 


6 


0 


3.14 


0.83 


0.69 


3 


(0.31) 


10b 


0 


1 


6 


3 


4 


0 


2.71 


0.96 


0.92 


3 


0.12 


10c 


1 


0 


5 


6 


2 


0 


2.77 


0.70 


0.49 


3 


(0.99) 


lOd 


1 


1 


6 


4 


2 


0 


2.54 


0.84 


0.71 


2 


(0.42) 
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Cost/Price Analyst Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


26 


1 


13 


12 


6 


1 


2.79 


0.88 


0.77 


3 


0.20 


Validating 


19 


1 


13 


19 


7 


0 


2.80 


0.75 


0.56 


3 


(0.39) 


Program Est 


19 


5 


22 


12 


1 


0 


2.23 


0.69 


0.47 


2 


(0.22) 


Budgeting 


14 


7 


23 


13 


2 


0 


2.22 


0.76 


0.57 


2 


(0.31) 


ICE 


20 


7 


13 


11 


8 


0 


2.51 


1.01 


1.02 


2 


0.09 


Software 


20 


6 


11 


15 


7 


0 


2.59 


0.95 


0.91 


3 


(0.05) 


Repair Parts 


18 


2 


9 


21 


9 


0 


2.90 


0.79 


0.62 


3 


(0.49) 


R&D 


14 


5 


13 


13 


12 


2 


2.84 


1.07 


1.15 


3 


(0.24) 


Service 


16 


2 


4 


19 


18 


0 


3.23 


0.80 


0.64 


3 


(0.77) 


Hardware 


14 


5 


10 


24 


6 


0 


2.69 


0.84 


0.70 


3 


(0.60) 


Construction 


22 


4 


9 


14 


10 


0 


2.81 


0.95 


0.91 


3 


(0.06) 


Commodity 


25 


1 


5 


14 


14 


0 


3.21 


0.80 


0.63 


3 


(0.14) 


















































Question 2 
























CE 


15 


10 


17 


10 


6 


1 


2.34 


1.03 


1.06 


2 


0.18 


PD&RR 


13 


8 


19 


10 


8 


1 


2.46 


1.04 


1.07 


2 


0.05 


E&MD 


11 


6 


19 


18 


5 


0 


2.46 


0.82 


0.68 


2 


(0.49) 


PFD&OS 


12 


7 


13 


20 


7 


0 


2.57 


0.92 


0.84 


3 


(0.45) 


















































Question 3 
























FFP 


12 


9 


12 


18 


7 


1 


2.55 


1.03 


1.06 


3 


(0.20) 


FPE 


21 


5 


5 


18 


7 


1 


2.84 


0.97 


0.95 


3 


(0.05) 


FPIS 


23 


4 


10 


10 


3 


0 


2.58 


0.79 


0.63 


3 


(0.09) 


FPR(P) 


20 


5 


0 


21 


4 


0 


2.62 


0.84 


0.70 


3 


(0.24) 


FPR(R) 


21 


5 


7 


20 


6 


0 


2.71 


0.89 


0.70 


3 


(0.17) 


FPIF 


20 


5 


10 


18 


6 


0 


2.64 


0.89 


0.78 


3 


(0.16) 


CPIF 


15 


5 


10 


20 


0 


0 


2.75 


0.91 


0.82 


3 


(0.46) 


CPAF 

CPFF 


17 


e 


12 


16 


7 


1 


2.64 


1.00 


0.95 


3 


(0.11) 


16 


7 


13 


15 


7 


1 


2.58 


1.02 


1.03 


3 


(0.07^ 


T&M 


18 


3 


4 


17 


13 


4 


3.27 


1.01 


1.03 


3 


(0.37) 


-abor Hour 


20 


3 


6 


14 


12 


4 


3.21 


1.07 


1.14 


3 


(0.17) 


























Question 4 
























Pre-Award 


10 


11 


17 


15 


6 


0 


2.33 


0.96 


0.91 


2 


(0.15) 


jPost-Award 


13 


8 


7 


17 


13 


1 


2.83 


1.09 


1.19 


3 


(0.-37) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew/ 


5a 


9 


8 


15 


16 


11 


0 


2.60 


1.00 


1.00 


3 


(0.36) 


5b 


14 


2 


19 


18 


6 


0 


2.62 


0.77 


0.59 


3 


(0.53) 


5c 


11 


24 


13 


4 


0 




1.51 


0.86 


0.73 


2 


(0.03) 


5d 


8 


9 


28 


10 


3 


1 


2.20 


0.86 


0.75 


2 


0.25 


5e 


18 


1 


11 


16 


13 


0 


3.00 


0.83 


0.68 


3 


(0.45) 


5f 


15 


6 


19 


14 


5 


0 


2.41 


0.86 


0.74 


2 


(0.22) 


Question 6 
























6a 


16 


4 


24 


12 


3 


0 


2.33 


0.74 


0.55 


2 


(0.28) 


6b 


12 


2 


14 


20 


11 


0 


2.85 


0.82 


0.68 


3 


(0.72) 


6c 


14 


5 


25 


12 


3 


0 


2.29 


0.75 


0.56 


2 


(0.31) 


6d 


9 


4 


19 


22 


5 


0 


2.56 


0.78 


0.61 


3 


(0.70) 


6e 


11 


7 


16 


20 


5 


0 


2.48 


0.87 


0.75 


3 


(0.48) 


6f 


13 


6 


26 


10 


4 


0 


2.26 


0.79 


0.63 


2 


(0.19) 


6g 


5 


20 


13 


5 


0 


0 


1.61 


0.84 


0.71 


2 


(0.19) 


Question 7 
























7a 


8 


20 


27 


2 


2 


0 


1.73 


0.72 


0.51 


2 


0.48 


7b 


8 


18 


29 


4 


0 


0 


1.73 


0.60 


0.36 


2 


(0.36) 


7c 


9 


18 


31 


1 


0 


0 


1.66 


0.51 


0.26 


2 


(0.73) 


Question 8 


















' 






8a 


9 


6 


29 


7 


8 


0 


2.34 


0.89 


0.78 


2 


0.01 


8b 


9 


6 


34 


7 


3 


0 


2.14 


0.69 


0.48 


2 


(0.19) 


8c 


8 


16 


27 


5 


3 


0 


1.90 


0.80 


0.64 


2 


0.35 


8d 


15 


9 


25 


6 


4 


0 


2.11 


0.83 


0.69 


2 


0.10 


























Question 9 
























9 


16 


4 


17 


14 


7 


1 


2.63 


0.94 


0.88 


3 


(0.15) 


Question 10 
























10a 


11 


3 


19 


22 


4 


0 


2.56 


0.73 


0.54 


3 


(0.75) 


10b 


20 


0 


11 


22 


6 


0 


2.87 


0.65 


0.42 


3 


(0.46) 


10c 


11 


3 


20 


20 


5 


0 


2.56 


0.76 


0.58 


3 


(0.65) 


lOd 


12 


10 


12 


19 


6 


0 


2.45 


0.96 


0.93 


3 


(0.30) 
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Contract Negotiator Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


5 


0 


1 


2 


0 


0 


2.67 


0.47 


0.22 


3 


0.81 


Validating 


4 


0 


2 


2 


0 


0 


2.50 


0.50 


0.25 


3 


0.27 


Program Est 


4 


2 


2 


0 


0 


0 


1.50 


0.50 


0.25 


2 


0.62 


Budgeting 


4 


3 


1 


0 


0 


0 


1.25 


0.43 


0.19 


1 


0.82 


ICE 


4 


1 


0 


3 


0 


0 


2.50 


0.87 


0.75 


3 


0.48 


Software 


3 


2 


0 


3 


0 


0 


2.20 


0.98 


0.96 


3 


0.34 


Repair Parts 


4 


1 


1 


2 


0 


0 


2.25 


0.83 


0.69 


3 


0.62 


R&D 


2 


2 


1 


1 


2 


0 


2.50 


1.26 


1.58 


3 


026 


Service 


2 


1 


3 


1 


1 


0 


2.33 


0.94 


0.89 


2 


0.16 


Hardware 


3 


2 


2 


1 


0 


0 


1.80 


0.75 


0.56 


2 


0.49 


Construction 


4 


1 


3 


0 


0 


0 


1.75 


0.43 


0.19 


2 


0.31 


Commodity 


5 


0 


0 


1 


2 


0 


3.67 


0.47 


0.22 


4 


0.74 


















































Question 2 
























CE 


2 


1 


2 


2 


1 


0 


2.50 


0.96 


0.92 


3 


(0.09) 


PD&RR 


2 


2 


1 


2 


1 


0 


2.33 


1.11 


1.22 


3 


0.22 


E&MD 


1 


2 


2 


2 


1 


0 


2.29 


1.03 


1.06 


2 


0.00 


PFD&OS 


2 


2 


1 


1 


2 


0 


2.50 


1.26 


1.58 


3 


0.26 


















































Question 3 
























FFP 


2 


1 


1 


4 


0 


0 


2.50 


0.76 


0.58 


3 


(0.62) 


FPE 


4 


1 


1 


2 


0 


0 


2.25 


0.83 


0.69 


3 


0.62 


FPIS 


4 


1 


1 


2 


0 


0 


2.25 


0.83 


0.69 


3 


0.62 


FPR(P) 


4 


1 


1 


2 


0 


0 


2.25 


0.83 


0.69 


3 


0.62' 


FPR(R) 


4 


1 


1 


2 


0 


0 


2.25 


0.83 


0.69 


3 


0.62 


FPIF 


4 


1 


1 


2 


0 


0 


2.25 


0.83 


0.69 


3 


0.62: 


CPIF 


4 


2 


2 


0 


0 


0 


1.50 


0.50 


0.25 


2 


0.62 


CPAF 


3 


2 


2 


1 


0 


0 


1.80 


0.75 


0.56 


2 


0.49 


CPFF 


3 


2 


2 


O' 


1 


0 


2.00 


1.10 


1.20 


2 


1.12 


T&M 


3 


1 


2 


2 


0 


0 


2.20 


0.75 


0.56 


2 


0.11 


Labor Hour 


4 


1 


2 


0 


1 


0 


2.25 


1.09 


1.19 


2 


1.19 


























Question 4 
























Pre-Award 


1 


1 


2 


2 


2 


0 


2.71 


1.03 


1.06 


3 


(0.48) 


[Post-Award 


2 


1 


2 


1 


1 


1 


2.83 


1.34 


1.81 


3 


0.34 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


2 


0 


3 


3 


0 


0 


2.50 


0.50 


0.25 


3 


(0.88) 


5b 


2 


1 


4 


0 


1 


0 


2.17 


0.90 


0.81 


2 


0.41 


5c 


1 


3 


2 


2 


0 


0 


1.86 


0.83 


0.69 


2 


0.04 


5d 


1 


1 


4 


2 


0 


0 


2.14 


0.64 


0.41 


2 


(0.86) 


5e 


3 


1 


2 


1 


1 


0 


2.40 


1.02 


1.04 


2 


0.50 


5f 


2 


0 


6 


0 


0 


0 


2.00 


0.00 


0.00 


2 


(1.44) 


Question 6 
























6a 


1 


1 


5 


1 


0 


0 


2.00 


0.53 


0.29 


2 


(1.03) 


6b 


2 


1 


2 


2 


1 


0 


2.50 


0.96 


0.92 


3 


(0.09) 


6c 


2 


1 


2 


2 


1 


0 


2.50 


0.96 


0.92 


3 


(0.09) 


6d 


2 


2 


4 


0 


0 


0 


1.67 


0.47 


0.22 


2 


(0.62) 


6e 


2 


2 


4 


0 


0 


0 


1.67 


0.47 


0.22 


2 


(0.62) 


6f 


2 


3 


0 


3 


0 


0 


2.00 


1.00 


1.00 


2 


0.25 


6g 


3 


1 


2 


2 


0 


0 


2.20 


0.75 


0.56 


2 


0.11 


Question 7 
























7a 


2 


2 


3 


0 


1 


0 


2.00 


1.00 


1.00 


2 


0.76 


7« 


2 


2 


3 


1 


0 


0 


1.83 


0.64 


0.41 


2 


(0.27^ 


7c 


2 


2 


3 


1 


0 


0 


1.83 


0.69 


0.47 


2 


(0.04) 


Question 8 
























8a 


2 


1 


2 


2 


1 


0 


2.50 


0.96 


0.92 


3 


(0.09) 


8b 


2 


1 


2 


2 


1 


0 


2.50 


0.96 


0.92 


3 


(0.09) 


8c 


2 


0 


4 


1 


1 


0 


2.50 


0.76 


0.58 


2 


(0.16) 


8d 


2 


0 


4 


1 


1 


0 


2.50 


0.76 


0.58 


2 


(0.16) 


























Question 9 
























9 


2 


0 


3 


2 


1 


0 


2.67 


0.75 


0.56 


3 


(0.40) 


Question 10 
























10a 


1 


0 


5 


1 


1 


0 


2.43 


0.73 


0.53 


2 


(0.31) 


10b 


1 


0 


3 


2 


2 


0 


2.86 


0.83 


0.69 


3 


(0.76) 


10c 


2 


1 


2 


3 


0 


0 


2.33 


0.75 


0.56 


3 


(0.47) 


lOd 


2 


1 


3 


1 


1 


0 


2.33 


0.94 


0.89 


2 


0.16 
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Contracting Officer Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


18 


1 


11 


16 


3 


0 


2.68 


0.69 


0.48 


3 


(0.11) 


Validating 


13 


2 


11 


21 


2 


0 


2.64 


0.67 


0.45 


3 


(0.52) 


Program Est 


12 


3 


18 


15 


1 


0 


2.38 


0.67 


0.45 


2 


(0.44) 


Budgeting 


13 


5 


18 


11 


2 


0 


2.28 


0.77 


0.59 


2 


(0.13) 


ICE 


22 


1 


14 


10 


2 


0 


2.48 


0.69 


0.47 


2 


0.24 


Software 


22 


4 


10 


8 


5 


0 


2.52 


0.96 


0.92 


2 


0.46 


Repair Parts 


16 


0 


8 


22 


3 


0 


2.85 


0.56 


0.31 


3 


(0.43) 


R&D 


17 


3 


11 


11 


6 


1 


2.72 


0.98 


0.95 


3 


0.14 


Service 


16 


1 


8 


11 


12 


1 


3.12 


0.91 


0.83 


3 


(0.17) 


Hardware 


9 


0 


17 


20 


3 


0 


2.65 


0.61 


0.38 


3 


(0.81) 


Construction 


25 


0 


10 


9 


5 


0 


2.79 


0.76 


0.58 


3 


0.44 


Commodity 


26 


0 


3 


12 


8 


0 


3.22 


0.66 


0.43 


3 


0.34 


















































Question 2 
























CE 


12 


8 


6 


15 


7 


1 


2.65 


1.10 


1.20 


3 


(0.04) 


PD&RR 


12 


3 


11 


21 


2 


0 


2.59 


0.72 


0.51 


3 


(0.53) 


E&MD 


12 


2 


16 


17 


2 


0 


2.51 


0.68 


0.47 


3 


(0.47) 


PFD&OS 


11 


1 


15 


17 


5 


0 


2.68 


0.73 


0.53 


3 


(0.51) 


















































Question 3 
























FFP 


9 


4 


15 


15 


6 


0 


2.58 


0.86 


0.74 


3 


(0.39) 


FPE 


14 


1 


13 


14 


7 


0 


2.77 


0.80 


0.63 


3 


(0.27) 


FPIS 


16 


0 


13 


16 


4 


0 


2.73 


0.66 


0.44 


3 


(0.26) 


FPR(P) 


15 


0 


13 


17 


4 


0 


2.74 


0.66 


0.43 


3 


(0.35) 


FPR(R) 


16 


0 


13 


15 


5 


0 


2.76 


0.70 


0.49 


3 


(0.23) 


fPIF 


13 


0 


17 


15 


4 


0 


2.64 


0.67 


0.45 


3 


(0.40) 


iCPIF 


15 


2 


11 


16 


5 


0 


2.71 


0.79 


0.62 


3 


(0.23) 


jCPAF 


13 


2 


11 


16 


5 


0 


2.72 


0.77 


0.59 


3 


(0.39) 


bpFF 


13 


3 


9 


20 


4 


0 


2.69 


0.78 


0.60 


3 


(0.41) 


b&M 


15 


0 


8 


14 


11 


1 


3.15 


0.81 


0.65 


3 


(0.31) 


Labor Hour 


14 


0 


10 


12 


12 


1 


3.11 


0.85 


0.73 


3 


(0.32) 


























buestion 4 
























Pre-Award 


9 


3 


15 


19 


3 


0 


2.55 


0.74 


0.55 


3 


(0.61) 


Post-Award 


11 


0 


10 


14 


14 


0 


3.11 


0.79 


0.62 


3 


(0.63) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


10 


3 


17 


15 


4 


0 


2.51 


0.78 


0.61 


2 


(0.41) 


5b 


10 


2 


20 


14 


3 


0 


2.46 


0.71 


0.50 


2 


(0.46) 


5c 


8 


7 


24 


9 


1 


0 


2.10 


0.69 


0.48 


2 


(0.36) 


5d 


11 


1 


29 


5 


3 


0 


2.26 


0.64 


0.40 


2 


(0.25) 


5e 


11 


0 


5 


23 


10 


0 


3.13 


0.61 


0.38 


3 


(0.86) 


5f 


9 


0 


26 


13 


1 


0 


2.38 


0.53 


0.28 


2 


(0.78) 


Question 6 
























6a 


11 


6 


22 


6 


4 


0 


2.21 


0.83 


0.69 


2 


0.04 


6b 


9 


3 


12 


21 


4 


0 


2.65 


0.76 


0.58 


3 


(0.66) 


6c 


9 


4 


22 


10 


4 


0 


2.35 


0.79 


0.63 


2 


(0.24) 


6d 


11 


1 


21 


12 


4 


0 


2.50 


0.72 


0.51 


2 


(0.37) 


6e 


10 


4 


19 


14 


2 


0 


2.36 


0.73 


0.54 


2 


(0.41) 


6f 


10 


6 


29 


2 


2 


0 


2.00 


0.64 


0.41 


2 


(0.13) 


6g 


11 


2 


18 


14 


4 


0 


2.53 


0.75 


0.57 


2 


(0.38) 


Question 7 
























7a 


8 


10 


28 


3 


0 


0 


1.83 


0.54 


0.29 


2 


(0.64) 


7b 


8 


12 


27 


2 


0 


0 


1.76 


0.53 


0.28 


2 


(0.61) 


7c 


8 


14 


26 


1 


0 


0 


1.68 


0.52 


0.27 


2 


(0.61) 


Question 8 
























8a 


10 


7 


24 


7 


1 


0 


2.05 


0.68 


0.46 


2 


(0.25) 


8b 


10 


7 


25 


7 


0 


0 


2.00 


0.60 


0.36 


2 


(0.48) 


8c 


9 


11 


25 


4 


0 


0 


1.83 


0.59 


0.34 


2 


(0.43) 


8d 


12 


6 


27 


4 


0 


0 


1.95 


0.52 


0.27 


2 


(0.50) 


























Question 9 


16 


3 


23 


5 


2 


0 


2.18 


0.67 


0.45 


2 


0.04 


9 
























Question 10 
























10a 


11 


2 


16 


17 


3 


0 


2.55 


0.71 


0.51 


3 


(0.49) 


10b 


14 


0 


8 


22 


5 


0 


2.91 


0.60 


0.36 


3 


(0.55) 


10c 


12 


1 


7 


22 


7 


0 


2.95 


0.70 


0.48 


3 


(0.64) 


lOd 


11 


4 


19 


13 


2 


0 


2.34 


0.74 


0.54 


2 


(0.33) 
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Program Staff Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


3 


0 


0 


1 


0 


0 


3.00 


0.00 


0.00 


3 


2.00 


\/alidating 


1 


0 


0 


3 


0 


0 


3.00 


0.00 


0.00 


3 


(2.00) 


Program Est 


1 


0 


0 


3 


0 


0 


3.00 


0.00 


0.00 


3 


(2.00) 


Budgeting 


1 


0 


0 


3 


0 


0 


3.00 


0.00 


0.00 


3 


(2.00) 


ICE 


2 


0 


1 


1 


0 


0 


2.50 


0.50 


0.25 


3 


0.37 


Software 


1 


0 


1 


1 


1 


0 


3.00 


0.82 


0.67 


3 


(0.75) 


Repair Parts 


2 


0 


0 


1 


1 


0 


3.50 


0.50 


0.25 


4 


0.20 


R&D 


1 


0 


0 


2 


1 


0 


3.33 


0.47 


0.22 


3 


(1.54) 


Service 


2 


0 


0 


1 


1 


0 


3.50 


0.50 


0.25 


4 


0.20 


Hardware 


1 


0 


0 


2 


1 


0 


3.33 


0.47 


0.22 


3 


(1.54) 


Construction 


3 


0 


0 


1 


0 


0 


3.00 


0.00 


0.00 


3 


2.00 


Commodity 


3 


0 


0 


1 


0 


0 


3.00 


0.00 


0.00 


3 


2.00 


















































Question 2 
























CE 


2 


0 


2 


0 


0 


0 


2.00 


0.00 


0.00 


2 


0.00 


PD&RR 


2 


0 


2 


0 


0 


0 


2.00 


0.00 


0.00 


2 


0.00 


E&MD 


2 


0 


0 


2 


0 


0 


3.00 


0.00 


0.00 


3 


0.00 


PFD&OS 


0 


0 


0 


1 


3 


0 


3.75 


0.43 


0.19 


4 


(2.00) 


















































Question 3 
























FFP 


2 


0 


1 


1 


0 


0 


2.50 


0.50 


0.25 


3 


0.37 


FPE 


3 


0 


0 


1 


0 


0 


3.00 


0.00 


0.00 


3 


2.00 


FPIS 


3 


0 


0 


1 


0 


0 


3.00 


0.00 


0.00 


3 


2.00 


FPR(P) 


3 


0 


0 


1 


0 


0 


3.00 


0.00 


0.00 


3 


2.00 


IfPR{R) 


3 


0 


0 


1 


0 


0 


3.00 


0.00 


0.00 


3 


2.00 


:fpif 


2 


0 


1 


1 


0 


0 


2.50 


0.50 


0.25 


3 


0.37 


CPIF 


2 


0 


1 


0 


1 


0 


3.00 


1.00 


1.00 


3 


0.85 


CPAF 


2 


0 


1 


0 


1 


0 


3.00 


1.00 


1.00 


3 


0.85 


C-PFF 


2 


0 


1 


0 


1 


0 


3.00 


1.00 


j 1.00 


3 


0.85 


T&M 


2 


0 


0 


0 


2 


0 


4.00 


0.00 


0.00 


4 


0.00 


Labor Hour 


2 


0 


0 


0 


2 


0 


4.00 


0.00 


0.00 


4 


0.00 


























Question 4 
























Pre-Award 


1 


0 


0 


3 


0 


0 


3.00 


0.00 


0.00 


3 


(2.00) 


Post-Award 


1 


0 


0 


1 


2 


0 


3.67 


0.47 


0.22 


4 


(1.66) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


0 


0 


0 


1 


3 


0 


3.75 


0.43 


0.19 


4 


(2.00) 


5b 


1 


0 


0 


2 


1 


0 


3.33 


0.47 


0.22 


3 


(1.54) 


5c 


1 


0 


2 


0 


1 


0 


2.67 


0.94 


0.89 


2 


0.00 


5d 


0 


0 


2 


1 


1 


0 


2.75 


0.83 


0.69 


3 


0.85 


5e 


0 


0 


2 


0 


2 


0 


3.00 


1.00 


1.00 


3 


0.00 


5f 


0 


0 


3 


1 


0 


0 


2.25 


0.43 


0.19 


2 


2.00 


Question 6 
























6a 


1 


0 


1 


2 


0 


0 


2.67 


0.47 


0.22 


3 


(1.41) 


6b 


0 


0 


2 


2 


0 


0 


2.50 


0.50 


0.25 


3 


0.00 


6c 


0 


1 


1 


2 


0 


0 


2.25 


0.83 


0.69 


3 


(0.85) 


6d 


0 


0 


3 


1 


0 


0 


2.25 


0.43 


0.19 


2 


2.00 


6e 


0 


0 


2 


2 


0 


0 


2.50 


0.50 


0.25 


3 


0.00 


6f 


0 


2 


2 


0 


0 


0 


1.50 


0.50 


0.25 


2 


0.00 


6g 


1 


0 


1 


2 


0 


0 


2.67 


0.47 


0.22 


3 


(1.41) 


Question 7 
























7a 


0 


0 


2 


2 


0 


0 


2.50 


0.50 


0.25 


3 


0.00 


7b 


0 


1 


2 


1 


0 


0 


2.00 


0.71 


0.50 


2 


0.00 


7c 


0 


0 


4 


0 


0 


0 


2.00 


0.00 


0.00 


2 


0.00 


Question 8 
























8a 


0 


0 


4 


0 


0 


01 


2.00' 


0.00 


0.00 i 


2 


0.00 


8b 


0 


0 


4 


0 


0 


0 


2.00 


0.00 


0.00 


2 


0.00 


8c 


0 


0 


4 


0 


0 


0 


2.00 


0.00 


0.00 


2 


0.00 


8d 


0 


0 


4 


0 


0 


0 


2.00 


0.00 


0.00 


2 


0.00 


























Question 9 
























9 


2 


0 


1 


1 


0 


0 


2.50 


0.50 


0.25 


3 


0.37 


Question 10 
























10a 


1 


0 


0 


3 


0 


0 


3.00 


0.00 


0.00 


3 


(2.00) 


10b 


2 


0 


1 


1 


0 


0 


2.50 


0.50 


0.25 


3 


0.37 


10c 


0 


0 


3 


1 


0 


0 


2.25 


0.43 


0.19 


2 


2.00 


lOd 


2 


0 


0 


2 


0 


0 


3.00 


0.00 


0.00 


3 


0.00 
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Program Manager Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


1 


4 


1 


1 


0 


0 


1.50 


0.76 


0.58 


1 


0.86 


Validating 


1 


3 


1 


1 


1 


0 


2.00 


1.15 


1.33 


2 


0.71 


Program Est 


0 


6 


0 


1 


0 


0 


1.29 


0.70 


0.49 


1 


2.65 


Budgeting 


0 


4 


2 


1 


0 


0 


1.57 


0.73 


0.53 


1 


1.11 


ICE 


1 


3 


0 


3 


0 


0 


2.00 


1.00 


1.00 


2 


0.03 


Software 


1 


1 


3 


2 


0 


0 


2.17 


0.69 


0.47 


2 


(0.77) 


Repair Parts 


2 


1 


2 


1 


1 


0 


2.40 


1.02 


1.04 


2 


0.26 


R&D 


2 


0 


3 


1 


1 


0 


2.60 


0.80 


0.64 


2 


(0.11) 


Service 


1 


0 


0 


32 


0 


0 


3.00 


0.69 


0.47 


3 


(1.42) 


Hardware 


2 


2 


1 


3 


0 


0 


2.17 


0.89 


0.80 


2 


0.22 


Construction 


0 


2 


1 


4 


0 


0 


2.29 


0.88 


0.78 


3 


(0.76) 


Commodity 


2 


0 


0 


4 


1 


0 


3.20 


0.40 


0.16 


3 


(0.98) 


















































Question 2 
























CE 


0 


4 


1 


1 


1 


0 


1.86 


1.12 


1.27 


1 


1.15 


PD&RR 


0 


4 


1 


1 


1 


0 


1.86 


1.03 


1.06 


1 


1.78 


pMD 


0 


2 


2 


2 


1 


0 


2.29 


1.03 


1.06 


2 


0.25 


PFD&OS 


1 


2 


4 


0 


0 


0 


1.67 


0.90 


0.81 


3 


(0.60) 


















































Question 3 
























FFP 


1 


1 


2 


3 


0 


0 


2.33 


0.75 


0.56 


3 


(0.91) 


FPE 


3 


1 


1 


2 


0 


0 


2.25 


0.63 


0.69 


3 


0.36 


FPIS 


2 


1 


2 


2 


0 


0 


2.20 


0.75 


0.56 


2 


(0.22) 


FPR(P) 


2 


1 


2 


2 


0 


0 


2.20 


0.75 


0.56 


2 


(0.22) 


FPR(R) 


2 


1 


3 


1 


0 


0 


2.00 


0.63 


0.40 


2 


(0.24) 


FPIF 


2 


1 


3 


1 


0 


0 


2.00 


0.63 


0.40 


2 


(0.24) 


jCPIF 


2 


2 


2 


1 


0 


0 


1.80 


0.75 


0.56 


2 


0.25! 


iCPAF 


2 


2 


2 


1 


0 


0 


1.80 


0.75 


0.56 


2 


0.25 


CPFF 


2 


2 


2 


1 


0 


0 


1.80 


0.75 


0.56 


2 


0.25 


T&M 


1 


2 


1 


2 


1 


0 


2.33 


1.11 


1.22 


3 


0.00 


Labor Hour 


2 


0 


1 


3 


1 


0 


3.00 


0.63 


0.40 


3 


(0.68) 


























buestion 4 
























Pre-Award 


0 


4 


2 


0 


0 


0 


1.33 


0.73 


0.53 


1 


1.11 


Post-Award 


0 


2 


2 


2 


0 


0 


2.00 


0.83 


0.69 


2 


(0.35) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


0 


3 


2 


1 


1 


0 


2.00 


1.07 


1.14 


2 


0.91 


5b 


0 


3 


2 


0 


2 


0 


2.14 


1.25 


1.55 


2 


0.80 


5c 


0 


3 


2 


1 


1 


0 


2.00 


1.07 


1.14 


2 


0.91 


5d 


0 


4 


1 


2 


0 


0 


1.71 


0.88 


0.78 


1 


0.76 


5e 


0 


3 


3 


0 


1 


0 


1.86 


0.99 


0.98 


2 


1.52 


5f 


1 


3 


2 


0 


1 


0 


1.83 


1.07 


1.14 


2 


1.14 


Question 6 
























6a 


0 


4 


2 


0 


1 


0 


1.71 


1.03 


1.06 


1 


1.78 


6b 


0 


3 


1 


2 


1 


0 


2.14 


1.12 


1.27 


2 


0.41 


6c 


1 


1 


2 


2 


1 


0 


2.50 


0.96 


0.92 


3 


(0.35) 


6d 


0 


2 


4 


1 


0 


0 


1.86 


0.64 


0.41 


2 


0.17 


6e 


0 


2 


4 


1 


0 


0 


1.86 


0.64 


0.41 


2 


0.17 


6f 


0 


1 


6 


0 


0 


0 


1.86 


0.35 


0.12 


2 


(2.65) 


6g 


1 


0 


3 


1 


2 


0 


2.83 


0.90 


0.81 


3 


(0.57) 


Question 7 
























7a 


1 


3 


0 


0 


3 


0 


2.50 


1.50 


2.25 


3 


0.21 


7e 


0 


3 


2 


0 


2 


0 


2.14 


1.25 


1.55 


2 


0.80 


7c 


0 


4 


1 


0 


2 


0 


2.00 


1.31 


1.71 


1 


0.80 


Question 8 


















- 






6a 


2 


1 


2 


1 


1 


0 


2.40 


1.02 


1.04 


2 


0.26 


8b 


1 


1 


2 


1 


2 


0 


2.67 


1.11 


1.22 


3 


(0.26) 


8c 


1 


1 


5 


0 


0 


0 


1.83 


0.37 


0.14 


2 


(1.76) 


8d 


1 


0 


6 


0 


0 


0 


2.00 


0.00 


0.00 


2 


(2.65) 


























Question 9 
























9 


1 


2 


4 


0 


0 


0 


1.67 


0.47 


0.22 


2 


(1.11) 


Question 10 
























10a 


1 


1 


2 


1 


2 


0 


2.67 


1.11 


1.22 


3 


(0.26) 


10b 


1 


0 


2 


4 


0 


0 


2.67 


0.47 


0.22 


3 


(1.78) 


10c 


1 


1 


1 


3 


1 


0 


2.67 


0.94 


0.89 


3 


(0.71) 


lOd 


1 


2 


2 


1 


1 


0 


2.17 


1.07 


1.14 


2 


0.35 
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Engineer Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


4 


1 


2 


2 


3 


0 


2.88 


1.05 


1.11 


3 


0.02 


Validating 


3 


6 


1 


2 


0 


0 


1.56 


0.83 


0.69 


1 


0.81 


Program Est 


3 


4 


3 


2 


0 


0 


1.78 


0.79 


0.62 


2 


0.26 


Budgeting 


1 


7 


3 


1 


0 


0 


1.45 


0.66 


0.43 


1 


0.67 


ICE 


3 


6 


2 


1 


0 


0 


1.44 


0.68 


0.47 


1 


0.71 


Software 


2 


6 


1 


2 


1 


0 


1.80 


1.08 


1.16 


1 


0.85 


Repair Parts 


3 


2 


3 


4 


0 


0 


2.22 


0.79 


0.62 


2 


(0.29) 


R&D 


3 


4 


3 


2 


0 


0 


1.78 


0.79 


0.62 


2 


0.26 


Service 


2 


2 


0 


5 


3 


0 


2.90 


1.04 


1.09 


3 


(0.67) 


Hardware 


2 


4 


1 


5 


0 


0 


2.10 


0.94 


0.89 


3 


(0.16) 


Construction 


4 


2 


3 


3 


0 


0 


2.13 


0.78 


0.61 


2 


0.06 


Commodity 


5 


1 


1 


3 


2 


0 


2.86 


0.99 


0.98 


3 


0.21 


















































Question 2 
























CE 


1 


5 


3 


2 


1 


0 


1.91 


1.00 


0.99 


2 


0.58 


PD&RR 


1 


4 


5 


2 


0 


0 


1.82 


0.72 


0.51 


2 


(0.14) 


E&MD 


1 


5 


1 


5 


0 


0 


2.00 


0.95 


0.91 


2 


(0.09) 


PFD&OS 


1 


3 


3 


3 


2 


0 


2.36 


1.07 


1.14 


2 


(0.05) 


















































Question 3 
























FFP 


3 


3 


4 


2 


0 


0 


1.89 


0.74 


0.54 


2 


(0.00) 


FPE 


7 


3 


1 


1 


0 


0 


1.60 


0.80 


0.64 


1 


1.50 


FPIS 


7 


3 


1 


1 


0 


0 


1.60 


0.80 


0.64 


1 


1.50 


FPR(P) 


6 


3 


2 


1 


0 


0 


1.67 


0.75 


0.56 


2 


0.99 


!fpr(R) 


6 


3 


2 


1 


0 


0 


1.67 


0.75 


0.56 


2 


0.99 


FPIF 


4 


5 


2 


1 


0 


0 


1.50 


0.71 


0.50 


1 


0.76 


CPIF 


4 


3 


5 


0 


0 


0 


1.63 


0.48 


0.23 


2 


(0.18) 


CPAF 


5 


3 


4 


0 


0 


0 


1.57 


0.48 


0.24 


2 


0.18 


CPFF 


4 


3 


3 


2 


0 


0 


1.88 


0.78 


0.61 


2 


0.31 


T&M 


4 


1 


4 


1 


2 


0 


2.50 


1.00 


1.00 


2 


0.30 


Labor Hour 


4 


1 


4 


1 


2 


0 


2.50 


1.00 


1.00 


2 


0.30 


























Question 4 
























Pre-Award 


2 


4 


3 


3 


0 


0 


1.90 


0.83 


0.69 


2 


0.00 


Post-Award 


2 


2 


3 


4 


1 


0 


2.40 


0.92 


0.84 


3 


(0.31) 



Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


1 


2 


7 


2 


0 


0 


2.00 


0.60 


0.36 


2 


(0.77) 


5b 


3 


3 


3 


3 


0 


0 


2.00 


0.82 


0.67 


2 


0.00 


5c 


1 


6 


5 


0 


0 


0 


1.45 


0.50 


0.25 


1 


(0.44) 


5d 


0 


5 


6 


0 


0 


1 


1.83 


1.07 


1.14 


2 


2.28 


5e 


3 


1 


5 


3 


0 


0 


2.22 


0.63 


0.40 


2 


(0.49) 


5f 


3 


1 


7 


1 


0 


0 


2.00 


0.47 


0.22 


2 


(0.65) 


Question 6 
























6a 


1 


2 


5 


3 


1 


0 


2.27 


0.86 


0.74 


2 


(0.19) 


6b 


0 


1 


2 


5 


4 


0 


3.00 


0.91 


0.83 


3 


(0.76) 


6c 


3 


3 


5 


1 


0 


0 


1.78 


0.63 


0.40 


2 


(0.13) 


6d 


2 


2 


4 


3 


1 


0 


2.30 


0.90 


0.81 


2 


(0.16) 


6e 


3 


1 


4 


3 


1 


0 


2.44 


0.83 


0.69 


2 


(0.19) 


6f 


0 


1 


8 


2 


1 


0 


2.25 


0.72 


0.52 


2 


1.05 


6g 


3 


2 


2 


4 


1 


0 


2.44 


0.96 


0.91 


3 


(0.12) 


Question 7 
























7a 


1 


3 


6 


2 


0 


0 


1.91 


0.67 


0.45 


2 


(0.44) 


7b 


0 


3 


6 


3 


0 


6 


2.00 


0.71 


0.50 


2 


0.00. 


7c 


0 


4 


7 


1 


0 


0 


1.75 


0.60 


0.35 


2 


0.17. 


Question 8 


















* 






8a 


1 


2 


3 


5 


1 


0 


2.45 : 


0.89 


0.79 


3 


(0.58) 


8b 


2 


1 


4 


4 


1 


0 


2.50 


0.81 


0.65 


3 


(0.53) 


8c 


1 


3 


5 


3 


0 


0 


2.00 


0.74 


0.55 


2 


(0.41) 


8d 


5 


0 


4 


1 


2 


0 


2.71 


0.88 


0.78 


2 


0.33 


























Question 9 
























9 


3 


1 


5 


3 


0 


0 


2.22 


0.63 


0.40 


2 


(0.49) 


Question 10 
























10a 


0 


0 


7 


3 


1 


1 


2.67 


0.94 


0.89 


2 


1.50 


10b 


5 


0 


3 


3 


1 


0 


2.71 


0.70 


0.49 


3 


0.10 


10c 


1 


1 


5 


4 


1 


0 


2.45 


0.78 


0.61 


2 


(0.59) 


lOd 


1 


0 


7 


3 


1 


0 


2.45 


0.66 


0.43 


2 


(0.59) 
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Technical Support Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


1 


1 


1 


2 


0 


0 


2.25 


0.83 


0.69 


3 


(0.54) 


Validating 


1 


0 


2 


2 


0 


0 


2.50 


0.50 


0.25 


3 


(1.36) 


Program Est 


0 


1 


2 


1 


1 


0 


2.40 


1.02 


1.04 


2 


0.40 


Budgeting 


0 


0 


2 


3 


0 


0 


2.60 


0.49 


0.24 


3 


(0.61) 


ICE 


4 


1 


0 


0 


0 


0 


1.00 


0.00 


0.00 


1 


2.24 


Software 


1 


2 


1 


0 


0 


1 


2.25 


1.64 


2.69 


2 


1.52 


Repair Parts 


0 


0 


0 


3 


1 


0 


3.25 


0.49 


0.24 


3 


0.61 


R&D 


2 


0 


0 


1 


1 


1 


4.00 


0.82 


0.67 


4 


(0.20) 


Service 


1 


0 


1 


2 


1 


0 


3.00 


0.71 


0.50 


3 


(1.12) 


Hardware 


1 


0 


1 


2 


1 


0 


3.00 


0.71 


0.50 


3 


(1.12) 


Construction 


2 


0 


0 


1 


2 


0 


3.67 


0.47 


0.22 


4 


(0.44) 


Commodity 


3 


0 


0 


0 


2 


0 


4.00 


0.00 


0.00 


4 


0.61 


















































Question 2 
























CE 


1 


1 


2 


0 


1 


0 


2.25 


1.09 


1.19 


2 


0.55 


PD&RR 


0 


1 


1 


2 


0 


0 


2.25 


0.83 


0.69 


3 


(0.54) 


E&MD 


1 


0 


3 


1 


0 


0 


2.25 


0.43 


0.19 


2 


(1.29) 


PFD&OS 


1 


0 


1 


2 


1 


0 


3.00 


0.71 


0.50 


3 


(1.12) 


















































Question 3 
























FFP 


1 


0 


0 


2 


2 


0 


3.50 


0.50 


0.25 


4 


(1.74)1 


FPE 


2 


0 


0 


1 


2 


0 


3.67 


0.47 


0.22 


4 


(0.44) 


FPIS 


2 


0 


0 


2 


1 


0 


3.33 


0.47 


0.22 


3 


(0.38) 


FPR(P) 


2 


0 


0 


2 


1 


0 


3.33 


0.47 


0.22 


3 


(0.38) 


FPR(R) 


2 


0 


0 


2 


1 


0 


3.33 


0.47 


0.22 


3 


(0.38) 


FPIF 


2 


0 


0 


2 


1 


0 


3.33 


0.47 


0.22 


3 


(0.38) 


CPIF 


2 


0 


0 


3 


0 


0 


3.00 


0.00 


0.00 


3 


(0.61) 


CPAF 


2 


0 


0 


3 


0 


0 


3.00 


0.00 


0.00 


3 


(0.61) 


CPFF 


2 


0 


0 


3 


0 


0 


: 3.00 


0.00 


0.00 ' 


3 


(0.61^ 


T&M 


2 


0 


0 


2 


1 


0 


3.33 


0.47 


0.22 


3 


(0.38) 


Labor Hour 


2 


0 


0 


2 


1 


0 


3.33 


0.47 


0.22 


3 


(0.38) 


























Question 4 
























Pre-Award 


1 


0 


1 


3 


0 


0 


2.75 


0.43 


0.19 


3 


(1.71) 


Post-Award 


1 


0 


1 


2 


1 


0 


3.00 


0.71 


0.50 


3 


(1.12) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


1 


0 


2 


1 


1 


0 


2.75 


0.83 


0.69 


3 


(0.55) 


5b 


1 


0 


3 


1 


0 


0 


2.25 


0.43 


0.19 


2 


(1.29) 


5c 


2 


1 


2 


0 


0 


0 


1.67 


0.47 


0.22 


2 


0.00 


5d 


1 


1 


3 


0 


0 


0 


1.75 


0.43 


0.19 


2 


(1.26) 


5e 


2 


0 


1 


1 


1 


0 


3.00 


0.82 


0.67 


3 


0.05 


5f 


3 


0 


2 


0 


0 


0 


2.00 


0.00 


0.00 


2 


0.61 


Question 6 
























6a 


2 


0 


2 


1 


0 


0 


2.33 


0.47 


0.22 


2 


(0.17) 


6b 


3 


0 


1 


0 


1 


0 


3.00 


1.00 


1.00 


3 


1.26 


6c 


1 


1 


1 


2 


0 


0 


2.25 


0.83 


0.69 


3 


(0.54) 


6d 


2 


0 


2 


1 


0 


0 


2.33 


0.47 


0.22 


2 


(0.17) 


6e 


2 


0 


2 


1 


0 


0 


2.33 


0.47 


0.22 


2 


(0.17) 


6f 


1 


2 


2 


0 


0 


0 


1.50 


0.50 


0.25 


2 


(0.51) 


6g 


3 


0 


1 


1 


0 


0 


2.50 


0.50 


0.25 


3 


0.88 


Question 7 
























7a 


1 


1 


2 


1 


0 


0 


2.00 


0.71 


0.50 


2 


(0.40) 


i7e 


1 


1 


3 


0 


0 


0 


1.75 


0.43 


0.19 


2 


(1.26) 


.7c 


2 


1 


2 


0 


0 


0 


1.67 


0.47 


0.22 


2 


0.00 


i 

Question 8 
























8a 


2 


1 


2 


0 


1 


0 


2.25 


j 0.94 


0.89 


2 


0.51 


8b 


2 


0 


2 


0 


1 


0 


2.67 


0.94 


0.89 


2 


0.51 


8c 


2 


0 


3 


0 


0 


0 


2.00 


0.00 


0.00 


2 


(0.61) 


8d 


2 


0 


2 


1 


0 


0 


2.33 


0.47 


0.22 


2 


(0.17) 


























Question 9 
























9 


2 


0 


1 


0 


1 


1 


3.67 


1.25 


1.56 


4 


0.23 


Question 10 
























10a 


1 


0 


3 


0 


1 


0 


2.50 


0.87 


0.75 


2 


0.00 


10b 


1 


0 


1 


1 


0 


1 


3.33 


1.30 


1.69 


4 


(0.52) 


10c 


1 


0 


2 


1 


0 


1 


3.00 


1.22 


1.50 


3 


0.27 


lOd 


2 


0 


0 


1 


1 


1 


4.00 


0.82 


0.67 


4 


(0.20) 
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Auditor Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


14 


1 


8 


16 


7 


0 


2.91 


0.76 


0.58 


3 


(0.33) 


Validating 


10 


2 


12 


15 


5 


2 


2.81 


0.94 


0.88 


3 


(0.22) 


Program Est 


14 


1 


10 


18 


3 


0 


2.72 


0.67 


0.45 


3 


(0.37) 


Budgeting 


17 


1 


13 


11 


4 


0 


2.62 


0.76 


0.58 


3 


0.02 


ICE 


33 


1 


4 


4 


4 


0 


2.85 


0.95 


0.90 


3 


1.40 


Software 


22 


1 


6 


9 


8 


0 


3.00 


0.87 


0.75 


3 


0.30 


Repair Parts 


18 


1 


7 


12 


8 


0 


2.96 


0.82 


0.68 


3 


(0.01) 


R&D 


15 


0 


4 


9 


17 


1 


3.48 


0.76 


0.57 


4 


(0.41) 


Service 


16 


1 


8 


8 


13 


0 


3.10 


0.91 


0.82 


3 


(0.13) 


Hardware 


14 


2 


15 


10 


5 


0 


2.56 


0.83 


0.68 


2 


(0.07) 


Construction 


17 


1 


10 


14 


4 


0 


2.72 


0.74 


0.54 


3 


(0.07) 


Commodity 


23 


0 


3 


12 


7 


1 


3.26 


0.74 


0.54 


3 


0.28 


















































Question 2 
























CE 


14 


1 


8 


9 


14 


0 


3.13 


0.89 


0.80 


3 


(0.29) 


PD&RR 


12 


1 


10 


17 


6 


0 


2.82 


0.75 


0.56 


3 


(0.46) 


E&MD 


14 


1 


9 


18 


4 


0 


2.78 


0.70 


0.48 


3 


(0.37) 


PFD&OS 


15 


1 


13 


14 


3 


0 


2.61 


0.70 


0.50 


3 


(0.20) 


















































Question 3 
























FFP 


15 


0 


14 


12 


5 


0 


2.71 


0.73 


0.53 


3 


(0.17) 


FPE 


18 


0 


12 


11 


5 


0 


2.75 


0.74 


0.54 


3 


0.03 


FPIS 


18 


0 


10 


13 


5 


0 


2.82 


0.71 


0.50 


3 


(0.03) 


FPR(P) 


17 


0 


13 


12 


4 


0 


2.69 


0.70 


0.49 


3 


(0.06) 


IfPR(R) 


16 


0 


11 


14 


5 


0 


2.80 


0.70 


0.49 


3 


(0.17) 


iFPIF 


15 


0 


13 


14 


4 


0 


2.71 


0.68 


0.46 


3 


(0.23) 


ICPIF 


15 


0 


e 


16 


0 


0 


2.90 


0.69 


0.47 


3 


(0.30) 


CPAF 


16 


1 


e 


14 


7 


0 


2.90 


0.79 


0.62 


3 


(0.15) 


CPFF 


15 


1 


8 


16 


6 


0 


2.87 


0.75 


0.56 


3 


(0.26) 


T&M 


15 


0 


5 


11 


15 


0 


3.32 


0.74 


0.54 


3 


(0.39) 


Labor Hour 


16 


0 


4 


13 


13 


0 


3.30 


0.69 


0.48 


3 


(0.34) 


























Question 4 
























Pre-Award 


11 


1 


17 


15 


2 


0 


2.51 


0.65 


0.42 


2 


(0.51) 


Post-Award 


11 


0 


1 


12 


17 


5 


3.74 


0.73 


0.53 


4 


(0.79) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


8 


6 


8 


19 


5 


0 


2.61 


0.90 


0.82 


3 


(0.49) 


5b 


7 


5 


21 


10 


3 


0 


2.28 


0.78 


0.61 


2 


(0.28) 


5c 


7 


4 


26 


8 


1 


0 


2.15 


0.62 


0.39 


2 


(0.54) 


5d 


8 


2 


21 


13 


2 


0 


2.39 


0.67 


0.45 


2 


(0.57) 


5e 


12 


0 


8 


19 


7 


0 


2.97 


0.66 


0.44 


3 


(0.59) 


5f 


7 


4 


22 


9 


4 


0 


2.33 


0.80 


0.63 


2 


(0.25) 


Question 6 
























6a 


7 


3 


25 


9 


2 


0 


2.26 


0.67 


0.45 


2 


(0.46) 


6b 


10 


2 


11 


20 


3 


0 


2.67 


0.71 


0.50 


3 


(0.64) 


6c 


10 


3 


17 


9 


7 


0 


2.56 


0.90 


0.80 


2 


(0.18) 


6d 


12 


6 


20 


8 


0 


0 


2.06 


0.64 


0.41 


2 


(0.29) 


6e 


12 


6 


15 


11 


2 


0 


2.26 


0.82 


0.67 


2 


(0.08) 


6f 


9 


10 


20 


6 


1 


0 


1.95 


0.73 


0.54 


2 


(0.05) 


6g 


13 


1 


14 


14 


4 


0 


2.64 


0.73 


0.53 


3 


(0.30) 


Question 7 
























7a 


6 


18 


19 


2 


1 


0 


1.65 


0.69 


0.48 


2 


0.32 


7b 


6 


13 


25 


1 


1 


0 


1.75 


0.62 


0.39 


2 


(0.07) 


7c 


6 


18 


21 


0 


1 


0 


1.60 


0.62 


0.39 


2 


0.24 


Question 8 
























8a 


8 


11 


14 


9 


4 


0 


2.16 


0.96 


0.92 


2 


0.12 


8b 


8 


13 


11 


9 


5 


0 


2.16 


1.04 


1.08 


2 


0.22 


8c 


6 


15 


21 


3 


1 


0 


1.75 


0.70 


0.49 


2 


0.13 


8d 


6 


12 


19 


8 


1 


0 


1.95 


0.77 


0.60 


2 


(0.07) 


























Question 9 
























9 


16 


5 


17 


7 


1 


0 


2.13 


0.72 


0.52 


2 


0.10 


Question 1 0 
























10a 


8 


2 


21 


12 


3 


0 


2.42 


0.71 


0.51 


2 


(0.48) 


10b 


13 


0 


9 


22 


2 


0 


2.79 


0.54 


0.29 


3 


(0.62) 


10c 


10 


1 


17 


15 


3 


0 


2.56 


0.68 


0.47 


3 


(0.53) 


lOd 


10 


0 


15 


19 


2 


0 


2.64 


0.58 


0.34 


3 


(0-75) 



242 



Industry Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


14 


7 


6 


0 


2 


0 


1.80 


0.98 


0.96 


2 


1.31 


Validating 


10 


8 


5 


4 


1 


1 


2.05 


1.15 


1.31 


2 


0.93 


Program Est 


12 


11 


4 


2 


0 


0 


1.47 


0.70 


0.48 


1 


0.89 


Budgeting 


2 


17 


6 


3 


1 


0 


1.56 


0.83 


■ 0.69 


1 


1.08 


ICE 


5 


10 


7 


6 


1 


0 


1.92 


0.91 


0.83 


2 


0.26 


Software 


7 


13 


4 


3 


1 


1 


1.77 


1.13 


1.27 


1 


1.24 


Repair Parts 


9 


3 


5 


8 


4 


0 


2.65 


0.96 


0.93 


3 


(0.03) 


R&D 


7 


7 


9 


4 


2 


0 


2.05 


0.93 


0.86 


2 


0.32 


Service 


8 


2 


4 


7 


8 


0 


3.00 


0.98 


0.95 


3 


(0.30) 


Hardware 


8 


9 


5 


4 


3 


0 


2.05 


1.09 


1.19 


2 


0.58 


Construction 


5 


7 


7 


6 


4 


0 


2.29 


1.06 


1.12 


2 


0.10 


Commodity 


13 


1 


1 


5 


9 


0 


3.38 


0.86 


0.73 


4 


0.07 


















































Question 2 
























CE 


1 


13 


10 


4 


1 


0 


1.75 


0.83 


0.69 


2 


0.68 


PD&RR 


1 


13 


10 


3 


2 


0 


1.79 


0.90 


0.81 


2 


0.87 


E&MD 


2 


10 


9 


8 


0 


0 


1.93 


0.81 


0.66 


2 


(0.11) 


PFD&OS 


4 


8 


5 


9 


3 


0 


2.28 


1.03 


1.06 


2 


(0.02) 


















































Question 3 
























FFP 


7 


9 


6 


5 


1 


1 


2.05 


1.11 


1.23 


2 


0.72 


FPE 


11 


e 


5 


3 


1 


1 


2.00 


1.15 


1.33 


2 


1.10 


FPIS 


10 


e 


5 


5 


1 


0 


1.95 


0.94 


0.89 


2 


0.56 


FPR(P) 


10 


e 


6 


4 


1 


0 


1.89 


0.91 


0.8§ 


2 


0.60 


FPR(R) 


10 


0 


5 


4 


1 


0 


1.84 


0.93 


0.87 


2 


0.70 


FPIF 


9 


e 


5 


5 


1 


0 


1.90 


0.94 


0.89 


2 


0.55 


|CPIF 


9 


12 


3 


2 


3 


0 


1.89 


1.12 


1.26 


1 


1.08 


ICPAF 

Igpff 


9 


12 


3 


2 


3 


0 


1.89 


1.12 


1.26 


1 


1.08 


9 


13 


3 


2 


2 


0 


1.65 


1.01 


1.03 


1 


1.20 


k&M 


8 


4 


4 


5 


4 


3 


2.90 


1.44 


2.09 


3 


0.29 


Labor Hour 


11 


2 


4 


5 


4 


3 


3.11 


1.24 


1.54 


3 


0.30 


























Question 4 
























Pre-Award 


3 


13 


7 


6 


0 


0 


1.73 


0.81 


0.66 


2 


0.25 


Post-Award 


5 


9 


5 


5 


4 


1 


2.29 


1.24 


1.54 


2 


0.42 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


0 


14 


7 


8 


0 


0 


1.79 


0.85 


0.72 


2 


0.43 


5b 


10 


7 


8 


3 


1 


0 


1.89 


0.85 


0.73 


2 


0.54 


5c 


0 


16 


9 


4 


0 


0 


1.59 


0.72 


0.52 


1 


0.85 


5d 


0 


14 


11 


2 


0 


2 


1.79 


1.06 


1.13 


2 


1.90 


5e 


11 


3 


7 


5 


3 


0 


2.44 


0.96 


0.91 


2 


0.32 


5f 


9 


7 


10 


2 


1 


0 


1.85 


0.79 


0.63 


2 


0.45 


Question 6 
























6a 


3 


7 


11 


5 


3 


0 


2.15 


0.95 


0.90 


2 


0.14 


6b 


2 


4 


2 


16 


5 


0 


2.81 


0.90 


0.82 


3 


(1.00) 


6c 


12 


5 


8 


3 


1 


0 


2.00 


0.84 


0.71 


2 


0.58 


6d 


0 


5 


7 


13 


4 


0 


2.55 


0.93 


0.87 


3 


(0.29) 


6e 


1 


4 


11 


9 


4 


0 


2.46 


0.91 


0.82 


2 


(0.20) 


6f 


3 


5 


13 


6 


2 


0 


2.19 


0.83 


0.69 


2 


(0.13) 


6g 


12 


1 


7 


6 


3 


0 


2.65 


0.84 


0.70 


3 


0.21 


Question 7 
























7a 


3 


9 


8 


6 


3 


0 


2.12 


1.01 


1.03 


2 


0.21 


7« 


2 


13 


10 


3 


1 


0 


1.70 


0.81 


0.65 


2 


0.65 


7c 


3 


13 


11 


1 


1 


0 


1.62 


0.74 


0.54 


2 


0.70 


Question 8 
























8a 


4 


4 


12 


4 


5 


0 


2.40 


0.98 


0.96 


1 2 


(0.02) 


8b 


5 


4 


15 


2 


3 


0 


2.17 


0.85 


0.72 


2 


0.13 


8c 


4 


6 


14 


4 


1 


0 


2.00 


0.75 


0.56 


2 


(0.09) 


8d 


16 


1 


8 


3 


1 


0 


2.31 


0.72 


0.52 


2 


0.72 


























Question 9 
























9 


7 


2 


8 


8 


3 


1 


2.68 


0.97 


0.94 


3 


(0.06) 


Question 10 
























10a 


1 


4 


12 


8 


2 


2 


2.50 


1.05 


1.11 


2 


0.45 


10b 


10 


0 


7 


11 


1 


0 


2.68 


0.57 


0.32 


3 


(0.32) 


10c 


2 


3 


9 


12 


3 


0 


2.56 


0.83 


0.69 


3 


(0.65) 


lOd 


2 


3 


12 


8 


4 


0 


2.48 


0.88 


0.77 


2 


(0.31) 
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U.S. Army Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


20 


1 


10 


17 


3 


0 


2.71 


0.68 


0.46 


3 


(0.06) 


Validating 


16 


2 


6 


21 


6 


0 


2.89 


0.75 


0.56 


3 


(0.34) 


Program Est 


13 


4 


16 


17 


1 


0 


2.39 


0.71 


0.50 


2 


(0.39) 


Budgeting 


14 


3 


14 


18 


2 


0 


2.51 


0.72 


0.52 


3 


(0.35) 


ICE 


18 


3 


13 


15 


2 


0 


2.48 


0.74 


0.55 


3 


(0.06) 


Software 


18 


5 


8 


13 


7 


0 


2.67 


0.97 


0.95 


3 


0.09 


Repair Parts 


14 


1 


6 


24 


6 


0 


2.95 


0.66 


0.43 


3 


(0.58) 


R&D 


13 


5 


8 


16 


8 


1 


2.79 


1.00 


1.01 


3 


(0.17) 


Service 


10 


3 


9 


15 


13 


1 


3.00 


0.96 


0.93 


3 


(0.49) 


Hardware 


9 


2 


10 


25 


5 


0 


2.79 


0.71 


0.50 


3 


(0.84) 


Construction 


17 


1 


13 


13 


7 


0 


2.76 


0.81 


0.65 


3 


(0.11) 


Commodity 


24 


0 


4 


18 


5 


0 


3.04 


0.58 


0.33 


3 


0.09 


















































Question 2 
























CE 


11 


7 


11 


15 


5 


2 


2.60 


1.07 


1.14 


3 


0.01 


PD&RR 


11 


3 


17 


18 


2 


0 


2.48 


0.71 


0.50 


3 


(0.52) 


E&MD 


10 


2 


15 


21 


3 


0 


2.61 


0.69 


0.48 


3 


(0.67) 


PFD&OS 


10 


3 


11 


17 


10 


0 


2.83 


0.88 


0.78 


3 


(0.52) 


















































Question 3 
























FFP 


13 


4 


9 


15 


9 


1 


2.84 


0.99 


0.98 


3 


(0.2lJ 


FPE 


IT 


2 


7 


14 


6 


1 


3.00 


0.92 


0.85 


3 


(0.05)' 


FPIS 


19 


1 


6 


20 


5 


0 


2.91 


0.68 


0.46 


3 


(0.19) 


FPR(P) 


18 


1 


e 


21 


5 


0 


2.91 


0.67 


0.85 


3 


(0.27) 


FPR(R) 


18 


1 


6 


19 


7 


0 


2.97 


0.72 


0.51 


3 


(0.24> 


FPIF 


18 


1 


9 


16 


6 


0 


2.84 


0.75 


0.57 


3 


(0.09^ 


CPIF 


17 


2 


8 


16 


e 


0 


2.88 


0.83 


0.85 


3 


(0.1 7i 


CPAF 


It 


1 


12 


13 


e 


0 


2.82 


0.92 


0.87 


3 


(0.12)| 


CPFF 


18 


3 


10 


14 


6 


0 


2.70 


0.87 


0.76 


3 


(0.00) 


T&M 


16 


1 


6 


17 


9 


2 


3.14 


0.87 


0.75 


3 


(0.23) 


Labor Hour 


17 


1 


7 


12 


12 


2 


3.21 


0.93 


0.87 


3 


(0.15) 


























Question 4 
























Pre-Award 


8 


4 


10 


26 


3 


0 


2.65 


0.74 


0.55 


3 


(0.84) 


Post-Award 


10 


2 


5 


18 


15 


1 


3.20 


0.86 


0.74 


3 


(0.73) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


7 


3 


16 


15 


10 


0 


2.73 


0.89 


0.79 


3 


(0.52) 


5b 


8 


2 


19 


15 


7 


0 


2.63 


0.81 


0.65 


3 


(0.50) 


5c 


6 


6 


26 


7 


6 


0 


2.29 


0.86 


0.74 


2 


(0.04) 


5d 


7 


4 


27 


9 


4 


0 


2.30 


0.76 


0.57 


2 


(0.27) 


5e 


10 


2 


8 


17 


14 


0 


3.05 


0.85 


0.73 


3 


(0.68) 


5f 


8 


2 


24 


13 


4 


0 


2.44 


0.73 


0.53 


2 


(0.47) 


Question 6 
























6a 


10 


8 


19 


8 


6 


0 


2.29 


0.94 


0.89 


2 


0.06 


6b 


8 


6 


15 


13 


9 


0 


2.58 


0.97 


0.94 


3 


(0.28) 


6c 


6 


3 


23 


12 


7 


0 


2.51 


0.83 


0.69 


2 


(0.36) 


6d 


6 


3 


25 


13 


4 


0 


2.40 


0.74 


0.55 


2 


(0.46) 


6e 


7 


7 


21 


14 


2 


0 


2.25 


0.77 


0.60 


2 


(0.39) 


6f 


9 


11 


24 


4 


3 


0 


1.98 


0.80 


0.64 


2 


0.18 


6g 


9 


5 


18 


16 


3 


0 


2.40 


0.79 


0.62 


2 


(0.42) 


Question 7 
























7a 


6 


16 


25 


3 


1 


0 


1.76 


0.67 


0.45 


2 


0.04 


7b 


6 


16 


24 


4 


1 


0 


1.78 


0.70 


0.48 


2 


0.06 


7c 


6 


17 


27 


0 


1 


0 


1.67 


0.60 


0.36 


2 


(0.03) 


'Question 8 
























Ba 


7 


6 


30 


6 


•2 


0 


2.09 


0.67 i 


0.45 


2 


(0.30) 


8b 


6 


6 


30 


8 


1 


0 


2.09 


0.63 


0.39 


2 


(0.53) 


8c 


7 


8 


32 


2 


2 


0 


1.95 


0.64 


0.41 


2 


(0.16) 


8d 


11 


5 


30 


4 


1 


0 


2.03 


0.57 


0.32 


2 


(0.39) 


























Question 9 
























9 


18 


3 


12 


13 


5 


0 


2.61 


0.85 


0.72 


3 


0.03 


Question 10 
























10a 


7 


1 


17 


23 


3 


0 


2.64 


0.64 


0.41 


3 


(0.96) 


10b 


11 


0 


10 


23 


7 


0 


2.93 


0.65 


0.42 


3 


(0.77) 


10c 


8 


2 


14 


20 


7 


0 


2.74 


0.78 


0.61 


3 


(0.70) 


lOd 


10 


7 


13 


16 


5 


0 


2.46 


0.91 


0.83 


3 


(0-25) 
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DCMC Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 




Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


12 


0 


8 


9 


5 


1 


2.96 


0.86 


0.74 


3 


(0.04) 


Validating 


11 


1 


9 


11 


3 


0 


2.67 


0.75 


0.56 


3 


(0.22) 


Program Est 


10 


3 


13 


9 


0 


0 


2.24 


0.65 


0.42 


2 


(0.31) 


Budgeting 


9 


6 


12 


8 


0 


0 


2.08 


0.73 


0.53 


2 


(0.18) 


ICE 


18 


5 


4 


5 


3 


0 


2.35 


1.08 


1.17 


2 


0.85 


Software 


13 


3 


7 


8 


4 


0 


2.59 


0.94 


0.88 


3 


0.17 


Repair Parts 


10 


1 


7 


13 


4 


0 


2.80 


0.75 


0.56 


3 


(0.38) 


R&D 


10 


1 


10 


6 


7 


1 


2.88 


0.99 


0.99 


3 


(0.05) 


Service 


12 


1 


4 


10 


8 


0 


3.09 


0.83 


0.69 


3 


(0.22) 


Hardware 


11 


2 


12 


9 


1 


0 


2.38 


0.70 


0.48 


2 


(0.17) 


Construction 


18 


1 


7 


4 


5 


0 


2.76 


0.94 


0.89 


3 


0.59 


Commodity 


20 


0 


3 


5 


7 


0 


3.27 


0.77 


0.60 


3 


0.55 


















































Question 2 
























CE 


12 


5 


6 


5 


7 


0 


2.61 


1.11 


1.23 


3 


0.25 


PD&RR 


10 


4 


9 


6 


5 


1 


2.60 


1.10 


1.20 


2 


0.20 


E&MD 


10 


3 


13 


7 


2 


0 


2.32 


0.75 


0.56 


2 


(0.13) 


PFD&OS 


9 


3 


1 


1 


0 


0 


1.60 


0.84 


0.71 


3 


(0.20) 


















































Question 3 
























FFP 


8 


12 


15 


8 


2 


0 


2.00 


0.86 


0.74 


2 


0.01 


FPE 


16 


10 


7 


10 


2 


0 


2.14 


0.88 


0.78 


3 


0.41 


FPIS 


17 


9 


8 


11 


0 


0 


2.07 


0.68 


0.46 


3 


0.30 


FPR(P) 


14 


6 


11 


9 


0 


0 


2.12 


0.70 


0.49 


2 


0.08 


FPR(R) 


15 


6 


8 


10 


1 


0 


2.24 


0.80 


0.64 


3 


0.25 


FPIF 


13 


0 


e 


11 


1 


0 


2.26 


0.78 


0.61 


3 


0.051 


CPIF 


10 


7 


e 


12 


3 


0 


2.37 


0.90 


0.81 


3 


(0.11) 


CPAF 


13 


7 


4 


14 


2 


0 


2.41 


0.89 


0.79 


3 


0.03 


CPFF 


11 


7 


6 


14 


2 


0 


2.38 


0.87 


0.75 


3 


(0.12)1 


T&M 


12 


3 


9 


9 


5 


2 


2.79 


1.08 


1.17 


3 


0.13 


Labor Hour 


12 


3 


11 


7 


5 


2 


2.71 


1.12 


1.24 


3 


0.23 


























Question 4 
























Pre-Award 


8 


5 


12 


7 


3 


0 


2.30 


0.90 


0.80 


2 


(0.01) 


Post-Award 


9 


4 


6 


11 


4 


1 


2.69 


1.03 


1.06 


3 


(0.06) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 




Std Dev 


Variation 


Median 


Skew 


5a 


9 


3 


11 


9 


3 


0 


2.46 


0.84 


0.71 


2 


(0.16) 


5b 


10 


2 


14 


7 


2 


0 


2.36 


0.74 


0.55 


2 


(0.12) 


5c 


7 


8 


14 


5 


1 


0 


1.96 


0.78 


0.61 


2 


0.03 


5d 


7 


3 


17 


7 


1 


0 


2.21 


0.67 


0.45 


2 


(0.39) 


5e 


14 


0 


7 


10 


4 


0 


2.86 


0.71 


0.50 


3 


_ (0.01) 


5f 


9 


2 


16 


6 


2 


0 


2.31 


0.72 


0.52 


2 


(0.16) 


Question 6 
























6a 


11 


2 


17 


5 


0 


0 


2.13 


0.53 


0.28 


2 


(0.31) 


6b 


11 


1 


9 


10 


4 


0 


2.71 


0.79 


0.62 


3 


(0.18) 


6c 


10 


2 


15 


8 


0 


0 


2.24 


0.59 


0.34 


2 


(0.39) 


6d 


9 


3 


13 


8 


2 


0 


2.35 


0.78 


0.61 


2 


(0.16) 


6e 


9 


3 


12 


8 


3 


0 


2.42 


0.84 


0.71 


2 


(0.12) 


6f 


10 


4 


14 


6 


1 


0 


2.16 


0.73 


0.53 


2 


(0.05) 


6g 


12 


3 


10 


8 


2 


0 


2.39 


0.82 


0.67 


2 


0.07 


Question 7 
























7a 


6 


12 


15 


1 


1 


0 


1.69 


0.70 


0.49 


2 


0.31 


[7b 


6 


8 


17 


4 


0 


c 


1.86 


0.63 


0.39 


2 


(0.37) 


7c 


7 


0 


16 


3 


0 


0 


1.79 


0.62 


0.38 


2 


(0.26) 


Question 8 
























8a 


7 


2 


18 


5 


3 


0 


2.32 


0.76 


0.58 


2 


(0.18) 


8b 


8 


4 


17 


3 


3 


0 


2.19 


0.82 


0.67 


2 


0.09 


8c 


7 


9 


15 


3 


1 


0 


1.86 


0.74 


0.55 


2 


0.13 


8d 


9 


5 


15 


4 


2 


0 


2.12 


0.80 


0.64 


2 


0.11 


























Question 9 
























9 


10 


2 


12 


7 


4 


0 


2.52 


0.85 


0.73 


2 


(0.06) 


Question 10 
























10a 


8 


2 


12 


10 


3 


0 


2.52 


0.79 


0.62 


2 


(0.33) 


10b 


13 


0 


7 


12 


3 


0 


2.82 


0.65 


0.42 


3 


(0.16) 


10c 


6 


1 


15 


12 


1 


0 


2.45 


0.62 


0.39 


2 


(0.77) 


lOd 


7 


4 


11 


11 


2 


0 


2.39 


0.82 


0.67 


2 


(0-34) 
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DCAA Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


14 


1 


8 


17 


7 


0 


2.91 


0.75 


0.57 


3 


(0.37) 


Validating 


9 


2 


12 


17 


5 


2 


2.82 


0.91 


0.83 


3 


(0.32) 


Program Est 


14 


1 


10 


18 


4 


0 


2.76 


0.70 


0.49 


3 


(0.37) 


Budgeting 


17 


1 


13 


12 


4 


0 


2.63 


0.75 


0.57 


3 


(0.03) 


ICE 


34 


1 


4 


4 


4 


0 


2.85 


0.95 


0.90 


3 


1.43 


Software 


22 


1 


6 


9 


8 


1 


3.08 


0.93 


0.87 


3 


0.32 


Repair Parts 


18 


1 


7 


13 


8 


0 


2.97 


0.81 


0.65 


3 


(0.05) 


R&D 


15 


0 


4 


9 


17 


2 


3.53 


0.79 


0.62 


4 


(0.41) 


Service 


16 


1 


8 


8 


14 


0 


3.13 


0.91 


0.82 


3 


(0.16) 


Hardware 


14 


2 


15 


11 


5 


0 


2.58 


0.82 


0.67 


2 


(0.11) 


Construction 


17 


1 


10 


14 


5 


0 


2.77 


0.76 


0.58 


3 


(0.08) 


Commodity 


24 


0 


3 


12 


7 


1 


3.26 


0.74 


0.54 


3 


0.32 


















































Question 2 
























CE 


14 


1 


9 


9 


14 


0 


3.09 


0.90 


0.81 


3 


(0.29) 


PD&RR 


11 


1 


10 


19 


6 


0 


2.83 


0.73 


0.53 


3 


(0.58) 


E&MD 


14 


1 


9 


19 


4 


0 


2.79 


0.69 


0.47 


3 


(0.40) 


PFD&OS 


15 


1 


13 


14 


4 


0 


2.66 


0.73 


0.54 


3 


(0.20) 


















































Question 3 
























FFP 


15 


0 


14 


12 


6 


0 


2.75 


0.75 


0.56 


3 


(0.18)1 


FPE 


16 


0 


12 


11 


6 


0 


2.79 


0.76 


0.58 


3 


0.01 


FPIS 


18 


0 


10 


13 


6 


0 


2.86 


0.73 


0.5t 


3 


(0.05) 


FPR(P) 


17 


0 


13 


12 


5 


0 


2.73 


0.73 


0.5t 


3 


(0.07> 


FPR(R) 


16 


0 


11 


14 


6 


0 


2.84 


0.72 


0.52 


3 


(0.19) 


FPIF 


15 


0 


13 


14 


5 


0 


2.75 


0.71 


0.60 


3 


(0.24), 


CPIF 


18 


0 


9 


18 


6 


0 


2.91 


0.67 


0.45 


3 


(0.43) 


CPAF 


16 


1 


9 


15 


7 


0 


2.90 


0.78 


0.60 


3 


(0.19) 


CPFF 


15 


1 


8 


17 


6 


0 


2.88 


0.74 


0.55 


3 


(0.30) 


T&M 


15 


0 


5 


11 


16 


0 


3.34 


0.73 


0.54 


4 


(0.43) 


Labor Hour 


16 


0 


4 


13 


14 


0 


3.32 


0.69 


0.48 


3 


(0.37) 


























Question 4 
























Pre-Award 


10 


1 


17 


17 


2 


0 


2.54 


0.64 


0.41 


3 


(0.63) 


Post-Award 


11 


0 


1 


12 


18 


5 


3.75 


0.72 


0.52 


4 


(0.83) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


7 


6 


9 


19 


6 


0 


2.63 


0.91 


0.83 


3 


(0.53) 


5b 


6 


5 


22 


11 


3 


0 


2.29 


0.77 


0.60 


2 


(0.34) 


5c 


7 


4 


27 


8 


1 


0 


2.15 


0.61 


0.38 


2 


(0.56) 


5d 


7 


3 


22 


13 


2 


0 


2.35 


0.69 


0.48 


2 


(0.57) 


5e 


11 


0 


9 


20 


7 


0 


2.94 


0.66 


0.44 


3 


(0.68) 


5f 


7 


4 


23 


9 


4 


0 


2.33 


0.79 


0.62 


2 


(0.25) 


Question 6 
























6a 


6 


4 


25 


10 


2 


0 


2.24 


0.69 


0.48 


2 


(0.46) 


6b 


9 


2 


13 


20 


3 


0 


2.63 


0.70 


0.50 


3 


(0.70) 


6c 


9 


3 


17 


10 


8 


0 


2.61 


0.90 


0.82 


2 


(0.26) 


6d 


11 


6 


22 


8 


0 


0 


2.06 


0.62 


0.39 


2 


(0.39) 


6e 


11 


6 


17 


11 


2 


0 


2.25 


0.79 


0.63 


2 


(0.16) 


6f 


8 


12 


20 


6 


1 


0 


1.90 


0.74 


0.55 


2 


(0.01) 


6g 


13 


1 


14 


15 


4 


0 


2.65 


0.72 


0.52 


3 


(0.34) 


Question 7 
























7a 


5 


20 


19 


2 


1 


0 


1.62 


0.69 


0.47 


2 


0.42 


7b 


5 


15 


25 


1 


1 


0 


1.71 


0.63 


0.39 


2 


0.02 


7k 


5 


20 


21 


0 


1 


0 


1.57 


0.62 


0.39 


2 


0.34 


Question 8 
























Ba 


7 


12 


14 


9^ 


5 


0 


2.18 


1.00 


0.99 


2 


0.15 


8b 


7 


14 


11 


9 


6 


0 


2.18 


1.07 


1.14 


2 


0.22 


8c 


5 


16 


22 


3 


1 


0 


1.74 


0.69 


0.48 


2 


0.15 


8d 


5 


12 


21 


8 


1 


0 


1.95 


0.75 


0.57 


2 


(0.10) 


























Question 9 
























9 


15 


5 


17 


8 


1 


1 


2.25 


0.87 


0.75 


2 


0.31 


Question 10 
























10a 


7 


2 


23 


12 


3 


0 


2.40 


0.70 


0.49 


2 


(0.52) 


10b 


12 


0 


10 


22 


2 


1 


2.83 


0.65 


0.43 


3 


(0.52) 


10c 


9 


2 


17 


15 


3 


1 


2.58 


0.82 


0.66 


3 


(0.30) 


lOd 


9 


0 


15 


20 


2 


1 


2.71 


0.68 


0.47 


3 


(0.59) 
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NASA Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


3 


3 


1 


2 


0 


0 


1.83 


0.90 


0.81 


2 


0.57 


Validating 


1 


2 


3 


3 


0 


0 


2.13 


0.78 


0.61 


2 


(0.55) 


Program Est 


2 


5 


1 


0 


1 


0 


1.57 


1.05 


1.10 


1 


1.68 


Budgeting 


0 


4 


3 


2 


0 


0 


1.78 


0.79 


0.62 


2 


0.50 


ICE 


3 


3 


2 


1 


0 


0 


1.67 


0.75 


0.56 


2 


0.55 


Software 


3 


2 


2 


1 


1 


0 


2.17 


1.07 


1.14 


2 


0.64 


Repair Parts 


3 


1 


2 


2 


1 


0 


2.50 


0.96 


0.92 


3 


0.16 


R&D 


2 


3 


1 


1 


1 


0 


2.00 


1.12 


1.27 


2 


0.42 


Service 


1 


1 


2 


3 


2 


0 


2.75 


0.97 


0.94 


3 


(0.66) 


Hardware 


1 


3 


2 


2 


0 


0 


1.86 


1.05 


1.11 


2 


0.26 


Construction 


2 


1 


2 


4 


0 


0 


2.43 


0.88 


0.78 


3 


(0.56) 


Commodity 


3 


1 


1 


1 


3 


0 


3.00 


1.15 


1.33 


4 


0.00 


















































Question 2 
























CE 


1 


6 


1 


0 


1 


0 


1.50 


1.00 


1.00 


1 


1.92 


PD&RR 


2 


5 


1 


0 


1 


0 


1.57 


1.05 


1.10 


1 


1.68 


E&MD 


0 


1 


4 


4 


0 


0 


2.33 


0.67 


0.44 


2 


(0.61) 


PFD&OS 


1 


0 


2 


5 


1 


0 


2.88 


0.60 


0.36 


3 


(1.51) 


















































Question 3 
























FFP 


5 


1 


2 


1 


0 


0 


2.00 


0.71 


0.50 


2 


0.87 


^PE 


6 


1 


2 


0 


0 


0 


1.67 


0.47 


0.22 


2 


1.19 


fPIS 


6 


1 


2 


1 


0 


0 


2.00 


0.71 


0.50 


2 


0.87 


FPR(P) 


S 


1 


2 


0 


0 


0 


1.67 


0.47 


0.22 


2 


1.19 


FPR(R) 


6 


1 


2 


0 


0 


0 


1.67 


0.47 


0.22 


2 


1.19 


FPIF 


5 


1 


2 


1 


0 


0 


2.00 


0.71 


0.50 


2 


0.87 


CPIF 


5 


1 


2 


0 


1 


0 


2.25 


1.09 


1.19 


2 


1.36 


CPAF 


6 


1 


2 


0 


1 


0 


2.25 


1.09 


1.19 


2 


1.36 


CPFF 


5 


1 


2 


0 


1 


0 


2.25 


1.09 


1.19 


2 


1.36 


T&M 


6 


0 


1 


1 


1 


0 


3.00 


0.82 


0.67 


3 


1.22 


Labor Hour 


6 


0 


1 


1 


1 


0 


3.00 


0.82 


0.67 


3 


1.22 


























Question 4 
























Pre-Award 


1 


4 


1 


2 


0 


0 


1.71 


0.93 


0.86 


2 


0.15 


Post-Award 


2 


3 


1 


1 


2 


0 


2.29 


1.28 


1.63 


2 


0.47 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


1 


1 


3 


3 


1 


0 


2.50 


0.87 


0.75 


3 


(0.54) 


5b 


2 


1 


4 


2 


0 


0 


2.14 


0.64 


0.41 


2 


(0.54) 


5c 


1 


5 


2 


1 


0 


0 


1.50 


0.71 


0.50 


1 


0.66 


5d 


1 


5 


2 


1 


0 


0 


1.50 


0.71 


0.50 


1 


0.66 


5e 


2 


1 


3 


2 


1 


0 


2.43 


0.90 


0.82 


2 


(0.13) 


5f 


2 


0 


6 


1 


0 


0 


2.14 


0.35 


0.12 


2 


(1.07) 


Question 6 
























6a 


1 


3 


4 


1 


0 


0 


1.75 


0.66 


0.44 


2 


(0.21) 


6b 


0 


1 


1 


6 


0 


0 


2.63 


0.70 


0.48 


3 


(1.53) 


6c 


4 


1 


4 


0 


0 


0 


1.80 


0.40 


0.16 


2 


0.00 


6d 


1 


2 


3 


2 


1 


0 


2.25 


0.97 


0.94 


2 


0.00 


6e 


1 


0 


3 


3 


1 


0 


2.71 


0.78 


0.61 


3 


(0.93) 


6f 


1 


3 


3 


2 


0 


0 


1.88 


0.78 


0.61 


2 


(0.11) 


6g 


2 


0 


2 


4 


1 


0 


2.86 


0.64 


0.41 


3 


(0.86) 


Question 7 
























7a 


1 


4 


2 


1 


1 


0 


1.88 


1.05 


1.11 


2 


0.82 


7b 


1 


3 


4 


0 


0 


0 


1.57 


0.50 


0.25 


2 


(0.61 i 


7c 


1 


3 


4 


1 


0 


0 


1.75 


0.66 


0.44 


2 


(0.21^ 


Question 8 
Sa 
























2 


0 


4 


3 


0 


0 


2.43 


0.49 


0.24 


2 


(0.95) 


8b 


2 


0 


3 


4 


0 


0 


2.57 


0.49 


0.24 


3 


(1.05) 


8c 


2 


1 


4 


2 


0 


0 


2.14 


0.64 


0.41 


2 


(0.54) 


8d 


3 


1 


4 


1 


0 


0 


2.00 


0.58 


0.33 


2 


(0.15) 


























Question 9 
























9 


2 


1 


6 


0 


0 


0 


1.86 


0.35 


0.12 


2 


(1.19) 


Question 10 
























10a 


1 


0 


3 


2 


3 


0 


3.00 


0.87 


0.75 


3 


(0.88) 


10b 


2 


0 


2 


3 


2 


0 


3.00 


0.76 


0.57 


3 


(0.73) 


10c 


1 


1 


2 


3 


2 


0 


2.75 


0.97 


0.94 


3 


(0.66) 


lOd 


1 


1 


3 


3 


1 


0 


2.50 


0.87 


0.75 


3 


(0-54) 
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U.S. Navy Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


10 


0 


6 


5 


1 


0 


2.58 


0.64 


0.41 


3 


0.20 


Validating 


3 


1 


9 


8 


1 


0 


2.47 


0.68 


0.46 


2 


(0.79) 


Program Est 


3 


3 


9 


7 


0 


0 


2.21 


0.69 


0.48 


2 


(0.69) 


Budgeting 


3 


3 


12 


3 


1 


0 


2.11 


0.72 


0.52 


2 


(0.22) 


ICE 


8 


2 


5 


4 


3 


0 


2.57 


0.98 


0.96 


3 


0.23 


Software 


7 


1 


5 


6 


3 


0 


2.73 


0.85 


0.73 


3 


(0.12) 


Repair Parts 


7 


0 


5 


5 


5 


0 


3.00 


0.82 


0.67 


3 


(0.24) 


R&D 


5 


1 


3 


11 


2 


0 


2.82 


0.71 


0.50 


3 


(0.73) 


Service 


6 


0 


2 


8 


6 


0 


3.25 


0.66 


0.44 


3 


(0.66) 


Hardware 


2 


1 


9 


8 


2 


0 


2.55 


0.74 


0.55 


3 


(0.72) 


Construction 


12 


0 


1 


8 


1 


0 


3.00 


0.45 


0.20 


3 


0.32 


Commodity 


12 


0 


1 


6 


3 


0 


3.20 


0.60 


0.36 


3 


0.38 


















































Question 2 
























CE 


1 


5 


3 


9 


4 


0 


2.57 


1.05 


1.10 


3 


(0.45) 


PD&RR 


1 


2 


8 


9 


2 


0 


2.52 


0.79 


0.63 


3 


(0.61) 


E&MD 


0 


3 


6 


11 


2 


0 


2.55 


0.84 


0.70 


3 


(0.40) 


=>FD&OS 


1 


1 


7 


7 


6 


0 


2.86 


0.89 


0.79 


3 


(0.65) 


















































Question 3 
























FFP 


4 


0 


7 


8 


3 


0 


2.78 


0.71 


0.51 


3 


(0.71) 


FPE 


8 


0 


6 


5 


3 


0 


2.79 


0.77 


0.60 


3 


(0.03) 


FPIS 


9 


1 


5 


6 


1 


0 


2.54 


0.75 


0.56 


3 


0.11 


FPR(P) 


8 


1 


5 


7 


1 


0 


2.57 


0.73 


0.53 


3 


(0.08) 


FPR(R) 


9 


1 


6 


5 


1 


0 


2.46 


0.75 


0.56 


2 


0.17 


FPIF 


5 


0 


7 


6 


2 


0 


2.71 


0.67 


0.44 


3 


(0.61) 


CPIF 


6 


2 


6 


8 


2 


c 


2.50 


0.67 


0.75 


3 


(0.13) 


CPAF 


5 


2 


6 


e 


1 


0 


2.47 


0.78 


0.60 


3 


(0.43), 


CPFF 


4 


2 


6 


8 


2 


0 


2.56 


0.83 


0.69 


3 


(0.49) 


T&M 


9 


0 


1 


9 


3 


0 


3.15 


0.53 


0.28 


3 


(0.19) 


Labor Hour 


8 


0 


3 


7 


4 


0 


3.07 


0.70 


0.49 


3 


(0.22) 


























Question 4 
























Pre-Award 


2 


3 


9 


5 


3 


0 


2.40 


0.92 


0.84 


2 


(0.18) 


jPost-Award 


3 


1 


5 


6 


7 


0 


3.00 


0.92 


0.84 


3 


(0.76) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


1 


2 


9 


9 


1 


0 


2.43 


0.73 


0.53 


2 


(0.71) 


5b 


1 


2 


8 


10 


1 


0 


2.48 


0.73 


0.54 


3 


(0.83) 


5c 


0 


2 


13 


7 


0 


0 


2.23 


0.60 


0.36 


2 


(0.14) 


5d 


1 


2 


13 


3 


3 


0 


2.33 


0.84 


0.70 


2 


0.18 


5e 


3 


1 


4 


8 


6 


0 


3.00 


0.86 


0.74 


3 


(0.88) 


5f 


4 


1 


10 


6 


1 


0 


2.39 


0.68 


0.46 


2 


(0.55) 


Question 6 
























6a 


2 


1 


12 


5 


2 


0 


2.40 


0.73 


0.54 


2 


(0.40) 


6b 


1 


0 


7 


12 


2 


0 


2.76 


0.61 


0.37 


3 


(1.25) 


6c 


2 


2 


13 


3 


2 


0 


2.25 


0.77 


0.59 


2 


(0.10) 


6d 


1 


5 


7 


6 


3 


0 


2.33 


0.99 


0.98 


2 


(0.03) 


6e 


1 


5 


8 


5 


3 


0 


2.29 


0.98 


0.97 


2 


0.08 


6f 


4 


1 


14 


1 


2 


0 


2.22 


0.71 


0.51 


2 


(0.08) 


6g 


3 


1 


8 


5 


5 


0 


2.74 


0.91 


0.83 


3 


(0.47) 


Question 7 
























t7a 


2 


4 


12 


4 


0 


0 


2.00 


0.63 


0.40 


2 


(0.64) 


7e 


1 


5 


16 


0 


0 


0 


1.76 


0.43 


0.18 


2 


(1.67) 


7c 


1 


7 


14 


0 


0 


0 


1.67 


0.47 


0.22 


2 


(1.15) 


{Question 8 






















I 


8a 


6 


3 


7 


5 


1 


0I 


2.25 


0.83 


0.69 


2 


(0.02) 


8b 


6 


2 


10 


1 


3 


0 


2.31 


0.92 


0.84 


2 


0.24 


8c 


5 


4 


10 


1 


2 


0 


2.06 


0.87 


0.76 


2 


0.33 


8d 


6 


2 


11 


2 


1 


0 


2.13 


0.70 


0.48 


2 


(0.02) 


























Question 9 
























9 


8 


2 


10 


2 


0 


0 


2.00 


0.53 


0.29 


2 


(0.10) 


Question 10 
























10a 


3 


1 


7 


6 


5 


0 


2.79 


0.89 


0.80 


3 


(0.57) 


10b 


4 


1 


6 


9 


2 


0 


2.67 


0.75 


0.56 


3 


(0.69) 


10c 


5 


0 


6 


8 


3 


0 


2.82 


0.71 


0.50 


3 


(0.61) 


lOd 


5 


3 


6 


6 


2 


0 


2.41 


0.91 


0.83 


2 


(0-14) 
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USAF Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


5 


2 


5 


5 


0 


0 


2.25 


0.72 


0.52 


2 


(0.23) 


Validating 


1 


1 


7 


7 


1 


0 


2.50 


0.71 


0.50 


3 


(0.83) 


Program Est 


1 


5 


8 


3 


0 


0 


1.88 


0.70 


0.48 


2 


(0.24) 


Budgeting 


1 


5 


6 


4 


1 


0 


2.06 


0.90 


0.81 


2 


0.13 


CE 


3 


3 


7 


4 


0 


0 


2.07 


0.70 


0.49 


2 


(0.44) 


Software 


3 


2 


9 


3 


0 


0 


2.07 


0.59 


0.35 


2 


(0.66) 


Repair Parts 


5 


1 


1 


8 


2 


0 


2.92 


0.76 


0.58 


3 


(0.47) 


R&O 


3 


2 


9 


1 


2 


0 


2.21 


0.86 


0.74 


2 


0.13 


Service 


3 


1 


1 


7 


5 


0 


3.14 


0.83 


0.69 


3 


(0.95) 


Hardware 


2 


1 


5 


8 


1 


0 


2.60 


0.71 


0.51 


3 


(0.98) 


Construction 


7 


1 


4 


3 


2 


0 


2.60 


0.92 


0.84 


3 


0.31 


Commodity 


6 


1 


0 


3 


7 


0 


3.45 


0.89 


0.79 


4 


(0.32) 


















































Question 2 
























CE 


2 


4 


9 


1 


1 


0 


1.93 


0.77 


0.60 


2 


0.23 


PD&RR 


2 


5 


6 


2 


2 


0 


2.07 


1.00 


1.00 


2 


0.38 


E&MD 


2 


4 


8 


2 


1 


0 


2.00 


0.82 


0.67 


2 


0.15 


PFD&OS 


1 


4 


5 


6 


1 


0 


2.25 


0.85 


0.72 


2 


(0.48) 


















































Question 3 
























FFP 


0 


1 


4 


11 


1 


0 


2.71 


0.67 


0.44 


3 


(0.86) 


FPE 


3 


0 


3 


9 


2 


0 


2.93 


0.59 


0.35 


3 


(1.10) 


FPIS 


3 


0 


7 


5 


2 


0 


2.64 


0.72 


0.52 


3 


(0.60) 


FPR(P) 


3 


0 


7 


5 


2 


0 


2.64 


0.72 


0.52 


3 


(0.60) 


FPR(R) 


3 


0 


7 


5 


2 


0 


2.64 


0.72 


0.52 


3 


(0.60) 


FPIF 


3 


0 


e 


3 


2 


0 


2.50 


0.73 


0.54 


2 


(0.3t3 


*CPIF 


4 


2 


0 


3 


2 


0 


2.38 


0.92 


0.85 


2 


0.00 


CPAF 


2 


4 


7 


2 


2 


0 


2.13 


0.96 


0.92 


2 


0.25 


CPFF 


2 


4 


7 


1 


3 


0 


2.20 


1.05 


1.09 


2 


0.34 


T&M 


2 


1 


3 


2 


9 


0 


3.27 


1.00 


1.00 


4 


(1.02) 


Labor Hour 


2 


1 


3 


3 


8 


0 


3.20 


0.98 


0.96 


4 


(0.97) 


























Question 4 
























Pre-Award 


0 


4 


5 


7 


1 


0 


2.29 


0.89 


0.80 


2 


(0.12) 


Post-Award 


1 


2 


6 


4 


4 


0 


2.63 


0.96 


0.91 


3 


(0.37) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


0 


4 


4 


5 


4 


0 


2.53 


1.09 


1.19 


3 


(0.08) 


5b 


0 


2 


7 


7 


1 


0 


2.41 


0.77 


0.60 


2 


(0.10) 


5c 


0 


6 


6 


5 


0 


0 


1.94 


0.80 


0.64 


2 


0.12 


5d 


0 


6 


10 


1 


0 


0 


1.71 


0.57 


0.33 


2 


0.11 


5e 


1 


1 


3 


9 


3 


0 


2.88 


0.78 


0.61 


3 


(1.18) 


5f 


1 


3 


10 


2 


1 


0 


2.06 


0.75 


0.56 


2 


0.13 


Question 6 
























6a 


1 


4 


10 


2 


0 


0 


1.88 


0.60 


0.36 


2 


(0.56) 


6b 


1 


0 


2 


9 


5 


0 


3.19 


0.63 


0.40 


3 


(1.70) 


6c 


0 


5 


9 


3 


0 


0 


1.88 


0.68 


0.46 


2 


0.16 


6d 


2 


1 


7 


6 


1 


0 


2.47 


0.72 


0.52 


2 


(0.74) 


6e 


2 


1 


7 


5 


2 


0 


2.53 


0.81 


0.65 


2 


(0.52) 


6f 


0 


1 


12 


3 


1 


0 


2.24 


0.64 


0.42 


2 


1.16 


6g 


2 


1 


5 


6 


3 


0 


2.73 


0.85 


0.73 


3 


(0.69) 


Question 7 
























7a 


0 


3 


13 


1 


0 


0 


1.88 


0.47 


0.22 


2 


(0.40) 


7b 


0 


5 


11 


1 


0 


0 


1.76 


0.55 


0.30 


2 


(0.08) 


7c 


0 


5 


12 


0 


0 


0 


1.71 


0.46 


0.21 


2 


(0.99) 


Question 8 
























8a 


2 


4 


6 


2 


3 


0 


2.27 


1.06 


1.13 


2 


0.21 


8b 


2 


3 


8 


4 


0 


0 


2.07 


0.68 


0.46 


2 


(0.60) 


8c 


1 


7 


6 


2 


1 


0 


1.81 


0.88 


0.78 


2 


0.68 


8d 


1 


4 


8 


1 


3 


0 


2.19 


1.01 


1.03 


2 


0.44 


























Question 9 
























9 


1 


0 


11 


4 


1 


0 


2.38 


0.58 


0.33 


2 


(0.47) 


Question 10 
























10a 


0 


3 


3 


8 


3 


0 


2.65 


0.97 


0.93 


3 


(0.46) 


10b 


1 


0 


6 


6 


3 


1 


2.94 


0.90 


0.81 


3 


(0.32) 


10c 


1 


0 


3 


9 


4 


0 


3.06 


0.66 


0.43 


3 


(1.48) 


lOd 


1 


4 


6 


5 


1 


0 


2.19 


0.88 


0.78 


2 


(0.13) 
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Cost Estimating Community Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


10 


6 


10 


8 


6 


0 


2.47 


1.02 


1.05 


2 


0.05 


Validating 


2 


5 


12 


17 


3 


1 


2.55 


0.91 


0.83 


3 


<0.27) 


Program Est 


5 


8 


10 


16 


1 


0 


2.29 


0.85 


0.72 


2 


(0.47) 


Pudgeting 


4 


8 


13 


13 


2 


0 


2.25 


0.86 


0.74 


2 


._CQ.31) 


ICE 


15 


4 


9 


8 


4 


0 


2.48 


0.94 


0.89 


2 


0.25 


Software 


10 


6 


6 


11 


6 


1 


2.67 


1.11 


1.22 


3 


0.00 


Repair Parts 


11 


2 


4 


16 


7 


0 


2.97 


0.81 


0.65 


3 


(0.46) 


R&D 


10 


2 


11 


9 


8 


0 


2.77 


0.92 


0.85 


3 


(0.24) 


Service 


11 


1 


4 


14 


10 


0 


3.14 


0.78 


0.60 


3 


(0.52) 


Hardware 


9 


1 


10 


16 


4 


0 


2.74 


0.72 


0.51 


3 


(0.59) 


Construction 


8 


4 


10 


14 


4 


0 


2.56 


0.86 


0.75 


3 


(0.39) 


Commodity 


17 


1 


2 


13 


7 


0 


3.13 


0.74 


0.55 


3 


(0.02) 


















































Question 2 
























CE 


3 


10 


12 


6 


9 


0 


2.38 


1.11 


1.22 


2 


0.07 


PD&RR 


3 


7 


14 


9 


7 


0 


2.43 


1.00 


1.00 


2 


(0.11) 


E&MD 


5 


4 


12 


16 


3 


1 


2.58 


0.78 


0.61 


3 


(0.70) 


PFD&OS 


6 


2 


10 


20 


2 


0 


2.65 


0.68 


0.46 


3 


(0.93) 


















































Question 3 
























Iffp 


14 


5 


9 


14 


3 


0 


2.48 


0.78 


0.61 


3 


(0.09) 


Ifpe 


19 


4 


7 


11 


4 


0 


2.58 


0.79 


0.63 


3 


0.30 


FPIS 


19 


4 


e 


11 


2 


0 


2.42 


0.72 


0.52 


3 


0.29 


FPR(P) 


20 


4 


e 


10 


2 


0 


2.40 


0.7S 


0.54 


3 


0.40 


FPR(R) 


19 


4 


6 


11 


3 


0 


2.50 


0.76 


0.59 


3 


0.30 


FPIF 


17 


4 


0 


12 


4 


0 


2.57 


0.79 


0.63 


3 


0.12 


CPIF 


17 


5 


0 


10 


5 


0 


2.54 


0.90 


0.80 


3 


0.29 


CPAF 


17 


5 


0 


11 


4 


0 


2.50 


0.96 


0.73 


3 


0.21 


CPFF 


17 


5 


8 


11 


4 


0 


2.50 


0.86 


0.73 


3 : 


0.21 


T&M 


16 


5 


6 


11 


7 


0 


2.69 


0.91 


0.83 


3 


0.08 


Labor Hour 


17 


3 


8 


11 


6 


0 


2.71 


0.75 


0.56 


3 


0.08 


























Question 4 
























Pre-Award 


5 


9 


13 


15 


3 


1 


2.37 


0.87 


0.75 


2 


(0.42) 


Post-Award 


5 


6 


7 


14 


12 


2 


2.93 


1.00 


1.00 


3 


(0.66) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


5 


10 


8 


18 


4 


1 


2.46 


0.94 


0.88 


3 


(0.44) 


5b 


3 


8 


17 


14 


4 


0 


2.33 


0.85 


0.73 


2 


(0.34) 


5c 


5 


13 


17 


10 


0 


1 


2.00 


0.76 


0.57 


2 


(0.32) 


5d 


3 


13 


20 


8 


2 


0 


1.98 


0.82 


0.67 


2 


0.08 


5e 


8 


6 


11 


11 


9 


0 


2.62 


0.97 


0.94 


3 


(0.32) 


5f 


3 


9 


21 


8 


5 


0 


2.21 


0.89 


0.79 


2 


0.07 


Question 6 
























6a 


9 


9 


20 


5 


2 


0 


2.00 


0.78 


0.60 


2 


0.01 


6b 


5 


8 


10 


17 


5 


1 


2.54 


0.90 


0.82 


3 


(0.51) 


6c 


6 


2 


19 


11 


2 


0 


2.38 


0.69 


0.47 


2 


(0.57) 


6d 


4 


7 


16 


10 


4 


0 


2.30 


0.86 


0.74 


2 


(0.18) 


6e 


4 


6 


19 


9 


3 


0 


2.24 


0.78 


0.60 


2 


(0.27) 


6f 


9 


5 


22 


4 


0 


0 


1.97 


0.54 


0.29 


2 


(0.53) 


6g 


9 


1 


13 


13 


4 


0 


2.65 


0.74 


0.55 


3 


(0.47) 


Question 7 
























7a 


4 


14 


15 


6 


2 


0 


1.89 


0.80 


0.64 


2 


0.21 


i7b 


4 


13 


20 


2 


2 


0 


1.81 


0.68 


0.47 


2 


0.16 


7c 


5 


16 


17 


1 


1 


1 


1.72 


0.68 


0.46 


2 


0.38 


Question 8 
























8a 


9 


5 


15 


7 


4 


0 


2.32 


0.89 


0.80 


2 


j (0.00) 


8b 


9 


6 


16 


4 


5 


0 


2.26 


0.95 


0.90 


2 


0.19 


8c 


9 


11 


15 


2 


3 


0 


1.90 


0.89 


0.80 


2 


0.51 


8d 


11 


6 


16 


6 


1 


0 


2.07 


0.74 


0.55 


2 


0.00 


























Question 9 
























9 


14 


2 


12 


9 


3 


0 


2.50 


0.80 


0.63 


2 


0.04 


Question 10 
























10a 


3 


2 


13 


16 


7 


0 


2.74 


0.83 


0.68 


3 


(0.70) 


10b 


9 


1 


12 


15 


3 


0 


2.65 


0.70 


0.49 


3 


(0.56) 


10c 


8 


1 


12 


17 


2 


0 


2.63 


0.65 


0.42 


3 


(0.74) 


lOd 


11 


3 


11 


12 


3 


0 


2.52 


0.81 


0.66 


3 


(0.20) 
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Acquisition Community Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


35 


3 


21 


33 


7 




2.69 


0.74 


0.55 


3 


(0.10) 


Validating 


22 


8 


26 


32 


10 




2.58 


0.88 


0.78 


3 


(0.24) 


Program Est 


19 


12 


37 


29 


2 




2.26 


0.74 


0.54 


2 


(0.39) 


Budgeting 


19 


14 


35 


27 


5 




2.28 


0.82 


0.67 


2 


(0.22) 


ICE 


49 


9 


20 


17 


6 




2.38 


0.91 


0.82 


2 


0.63 


Software 


37 


8 


22 


19 


12 




2.57 


0.94 


0.88 


3 


0.21 


Repair Parts 


29 


3 


16 


38 


14 




2.89 


0.76 


0.58 


3 


(0.34) 


R&D 


22 


7 


19 


23 


27 




2.92 


1.02 


1.05 


3 


(0.29) 


Service 


24 


4 


17 


27 


27 




3.03 


0.92 


0.84 


3 


(0.39) 


Hardware 


16 


7 


34 


32 


11 




2.56 


0.82 


0.68 


3 


(0.43) 


Construction 


43 


2 


21 


27 


7 




2.68 


0.73 


0.53 


3 


0.17 


Commodity 


51 


1 


7 


23 


16 




3.15 


0.77 


0.60 


3 


0.36 


















































Question 2 
























CE 


•18 


13 


24 


28 


16 




2.58 


1.01 


1.02 


3 


(0.18) 


PD&RR 


17 


8 


33 


33 


10 




2.54 


0.83 


0.68 


3 


(0.41) 


E&MD 


15 


10 


36 


32 


7 




2.42 


0.80 


0.64 


2 


(0.42) 


PFD&OS 


14 


10 


32 


32 


11 




2.52 


0.86 


0.74 


3 


(0.42) 


















































Question 3 
























FFP 


17 


9 


32 


32 


11 




2.54 


0.85 


0.73 


3 


(0.37) 


FPE 


33 


4 


24 


26 


13 




2.72 


0.84 


0.71 


3 


(0.04) 


FPIS 


32 


4 


22 


34 


8 




2.68 


0.76 


0.57 


3 


(0.17) 


FPR(P) 


28 


5 


25 


35 


7 




2.61 


0.76 


0.57 


3 


(0.27) 


FPR(R) 


30 


6 


23 


32 


9 




2.63 


0.81 


0.66 


3 


(0.15) 


FPIF 


29 


4 


30 


28 


7 




2.55 


0.75 


0.56 


3 


(0.19) 


CPIF 


25 


8 


22 


24 


11 




2.58 


0.86 


0.74 


3 


(0.26) 


CPAF 


24 


10 


24 


28 


13 




2.59 


0.92 


0.85 


3 


(0.17) 


CPFF 


21 


11 


34 


33 


12 




2.51 


0.91 


0.83 


3 


(0.27^ 


fr&M 


26 


3 


18 


27 


26 




3.03 


0.91 


0.83 


3 


(0.32) 


Labor Hour 


26 


3 


29 


26 


24 




2.87 


0.92 


0.85 


3 


(0.29) 
























1 

1 


Question 4 






















1 


Pre-Award 


7 


14 


30 


34 


5 




2.36 


0.79 


0.63 


2 


(0.48)1 


Post-Award 


10 


5 


17 


27 


27 




3.00 


0.94 


0.88 


3 


(0.73)1 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


9 


11 


33 


34 


13 


12 


2.83 


0.88 


0.78 


3 


(0.46) 


5b 


10 


9 


44 


32 


5 


12 


2.68 


0.74 


0.54 


2 


(0.56) 


5c 


5 


6 


56 


16 


6 


8 


2.50 


0.77 


0.59 


2 


(0.02) 


5d 


8 


8 


65 


15 


4 


11 


2.47 


0.63 


0.40 


2 


(0.43;i 


5e 


12 


2 


26 


38 


21 


15 


3.21 


0.78 


0.61 


3 


(0.81) 


5f 


11 


7 


52 


24 


6 


14 


2.69 


0.72 


0.51 


2 


(0.45) 


Question 6 
























6a 


9 


11 


55 


20 


5 


12 


2.53 


0.72 


0.52 


2 


(0.34) 


6b 


13 


5 


27 


43 


12 


16 


3.07 


0.77 


0.60 


3 


(0.74) 


6c 


8 


10 


47 


23 


11 


11 


2.67 


0.83 


0.69 


2 


(0.25) 


6d 


12 


10 


52 


26 


4 


15 


2.64 


0.72 


0.52 


2 


(0.47) 


6e 


13 


11 


42 


27 


6 


16 


2.75 


0.78 


0.61 


2 


(0.37) 


6f 


11 


17 


55 


12 


5 


12 


2.41 


0.74 


0.55 


2 


(0.09) 


6g 


14 


5 


39 


32 


10 


17 


2.95 


0.77 


0.60 


2 


(0.52) 


Question 7 
























7a 


7 


28 


54 


9 


2 


10 


2.15 


0.68 


0.46 


2 


(0.06) 


7« 


6 


27 


57 


8 


1 


8 


2.10 


0.62 


0.38 


2 


(0.28) 


7c 


7 


30 


59 


3 


1 


10 


2.05 


0.57 


0.33 


2 


(0.35) 


Question 8 
























8a 


10 


16 


52 


16 


5 


13 


2.48 


0.76 


0.58 


2 


(0.19) 


8b 


10 


15 


55 


15 


5 


13 


2.48 


0.74 


0.54 


2 


(0.18) 


8c 


8 


23 


51 


5 


3 


11 


2.23 


0.65 


0.42 


2 


(0.09) 


8d 


13 


15 


60 


9 


2 


16 


2.45 


0.61 


0.38 


2 


(0.38) 


























Question 9 
























9 


28 


8 


43 


15 


6 


31 


3.09 


0.76 


0.58 


2 


0.02 


Question 10 
























10a 


10 


6 


43 


33 


7 


14 


2.81 


0.78 


0.61 


2 


(0.45) 


10b 


16 


0 


19 


55 


8 


19 


3.27 


0.61 


0.37 


3 


(0.94) 


10c 


10 


3 


31 


45 


11 


13 


3.00 


0.72 


0.52 


3 


(0.88) 


lOd 


12 


10 


36 


35 


7 


15 


2.82 


0.80 


0.63 


2 


(0.51) 
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Federal Government Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


66 


7 


39 


55 


16 


1 


2.70 


0.81 


0.65 


3 


(0.02) 


Validating 


43 


9 


47 


67 


16 


2 


2.68 


0.81 


0.66 


3 


(0.36) 


Program Est 


45 


21 


58 


54 


6 


0 


2.32 


0.78 


0.61 


2 


(0.25) 


Budgeting 


46 


22 


61 


47 


8 


0 


2.30 


0.80 


0.64 


2 


(0.15) 


ICE 


86 


17 


36 


33 


12 


0 


2.41 


0.91 


0.83 


2 


0.51 


Software 


68 


14 


38 


40 


23 


1 


2.65 


0.96 


0.92 


3 


0.17 


Repair Parts 


59 


5 


28 


66 


26 


0 


2.90 


0.76 


0.58 


3 


(0.28) 


R&D 


50 


12 


35 


45 


37 


5 


2.91 


1.02 


1.04 


3 


(0.14) 


Service 


50 


7 


27 


51 


48 


1 


3.07 


0.89 


0.79 


3 


(0.38) 


Hardware 


41 


11 


53 


64 


15 


0 


2.58 


0.78 


0.61 


3 


(0.42) 


Construction 


75 


5 


37 


46 


21 


0 


2.76 


0.81 


0.66 


3 


0.11 


Commodity 


91 


2 


12 


45 


33 


1 


3.20 


0.76 


0.57 


3 


0.25 


















































Question 2 
























CE 


43 


28 


39 


39 


33 


2 


2.59 


1.09 


1.18 


3 


0.00 


PD&RR 


39 


20 


51 


55 


18 


1 


2.51 


0.90 


0.82 


3 


(0.21) 


E&MD 


38 


14 


56 


64 


12 


0 


2.51 


0.77 


0.60 


3 


(0.45) 


PFD&OS 


39 


13 


47 


61 


24 


0 


2.66 


0.85 


0.73 


3 


(0.40) 


















































Question 3 
























FFP 


47 


12 


49 


54 


21 


1 


2.64 


0.87 


0.76 


3 


(0.21) 


FPE 


71 


6 


36 


48 


22 


1 


2.79 


0.85 


0.72 


3 


0.07 


FPIS 


73 


5 


38 


54 


14 


0 


2.69 


0.74 


0.55 


3 


0.03 


FPR(P) 


68 


6 


42 


55 


13 


0 


2.65 


0.75 


0.56 


3 


(0.04) 


FPR(R) 


69 


6 


39 


53 


17 


0 


2.70 


0.78 


0.61 


3 


(0.01) 


FPIF 


62 


5 


49 


52 


16 


0 


2.65 


0.76 


0.57 


3 


(0.11) 


CPIF 


58 


11 


38 


55 


22 


0 


2.70 


0.86 


0.73 


3 


(0.13) 


CPAF 


60 


13 


38 


51 


21 


1 


2.67 


0.90 


0.82 


3 


(0.02) 


CPFF 


57 


15 


38 


53 


20 


1 


2.64 


0.91 


0.83 


3 


(0.05) 


T&M 


62 


4 


22 


48 


43 


5 


3.19 


0.89 


0.79 


3 


(0.18); 


Labor Hour 


63 


4 


26 


42 


44 


5 


3.17 


0.92 


0.85 


3 


(0.12)| 


Question 4 














































Pre-Award 


30 


21 


56 


65 


12 


0 


2.44 


0.82 


0.68 


3 


(0.45) 


Post-Award 


37 


12 


24 


53 


51 


7 


3.12 


1.00 


1.01 


3 


(0.48) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


27 


19 


52 


60 


26 


0 


2.59 


0.90 


0.81 


3 


(0.43) 


5b 


29 


14 


74 


52 


15 


0 


2.44 


0.79 


0.62 


2 


(0.41) 


5c 


23 


31 


89 


33 


8 


0 


2.11 


0.76 


0.58 


2 


(0.18) 


5d 


25 


23 


91 


34 


11 


0 


2.21 


0.77 


0.59 


2 


(0.22) 


5e 


43 


5 


34 


66 


36 


0 


2.94 


0.80 


0.64 


3 


(0.54) 


5f 


33 


13 


89 


37 


12 


0 


2.32 


0.74 


0.55 


2 


(0.30) 


Question 6 
























6a 


33 


22 


88 


31 


10 


0 


2.19 


0.76 


0.58 


2 


(0.18) 


6b 


33 


10 


47 


71 


23 


0 


2.71 


0.80 


0.64 


3 


(0.59) 


6c 


33 


16 


81 


37 


17 


0 


2.36 


0.82 


0.67 


2 


(0.21) 


6d 


32 


20 


77 


44 


11 


0 


2.30 


0.79 


0.62 


2 


(0.29) 


6e 


33 


22 


68 


47 


14 


0 


2.35 


0.84 


0.70 


2 


(0.24) 


6f 


34 


32 


87 


23 


8 


0 


2.05 


0.76 


0.58 


2 


(0.04) 


6g 


43 


11 


58 


54 


18 


0 


2.56 


0.81 


0.66 


3 


(0.31) 


Question 7 
























7a 


22 


60 


86 


12 


4 


0 


1.75 


0.69 


0.48 


2 


0.12 


7b 


21 


54 


97 


10 


2 


0 


1.75 


0.62 


0.3§ 


2 


(0.21) 


7c 


22 


62 


94 


4 


2 


0 


1.67 


0.59 


0.35 


2 


(0.18) 


Question 8 
























8a 


33 


27 


80 


30 


14 


0 


2.21 


0.84 


0.71 


2 


(0.04) 


8b 


33 


29 


80 


29 


13 


0 


2.17 


0.84 


0.70 


2 


(0.02) 


8c 


29 


45 


89 


14 


7 


0 


1.89 


0.74 


0.55 


2 


0.10 


8d 


37 


29 


89 


21 


8 


0 


2.05 


0.74 


0.55 


2 


(0.04) 


























Question 9 
























9 


56 


13 


69 


34 


11 


1 


2.36 


0.81 


0.65 


2 


0.05 


Question 10 
























10a 


29 


9 


65 


61 


20 


0 


2.59 


0.78 


0.62 


3 


(0.54) 


10b 


46 


1 


41 


75 


19 


2 


2.86 


0.71 


0.50 


3 


(0.49) 


10c 


33 


6 


57 


67 


20 


1 


2.69 


0.77 


0.60 


3 


(0.55) 


lOd 


36 


19 


54 


61 


13 


1 


2.48 


0.85 


0.72 


3 


(Q:31) 
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Prepare Estimates Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


10 


8 


7 


7 


5 


0 


2.33 


1.09 


1.19 


2 


0.25 


Validating 


1 


10 


12 


12 


2 


0 


2.17 


0.90 


0.81 


2 


(0.03) 


Program Est 


2 


13 


11 


10 


1 


0 


1.97 


0.88 


0.77 


2 


0.09 


Budgeting 


2 


12 


14 


8 


1 


0 


1.94 


0.83 


0.68 


2 


0.12 


ICE 


9 


9 


6 


11 


2 


0 


2.21 


0.98 


0.95 


2 


0.08 


Software 


9 


10 


4 


11 


3 


0 


2.25 


1.06 


1.12 


3 


0.15 


Repair Parts 


11 


3 


6 


10 


7 


0 


2.81 


0.96 


0.92 


3 


(0.15) 


R&D 


10 


8 


6 


9 


4 


0 


2.33 


1.05 


1.11 


2 


0.18 


Service 


8 


2 


4 


14 


9 


0 


3.03 


0.85 


0.72 


3 


(0.65) 


Hardware 


7 


6 


5 


15 


4 


0 


2.57 


0.96 


0.91 


3 


(0.37) 


Construction 


11 


6 


4 


12 


4 


0 


2.54 


1.01 


1.02 


3 


(0.00) 


Commodity 


15 


2 


2 


11 


7 


0 


3.05 


0.88 


0.77 


3 


0.01 


















































Question 2 
























CE 


2 


17 


10 


5 


3 


0 


1.83 


0.97 


0.94 


2 


0.74 


PD&RR 


3 


14 


12 


5 


3 


0 


1.91 


0.95 


0.90 


2 


0.52 


E&MD 


3 


7 


9 


17 


1 


0 


2.35 


0.84 


0.70 


3 


(0.65) 


PFD&OS 


5 


3 


7 


16 


6 


0 


2.78 


0.84 


0.70 


3 


(0.76) 


















































Question 3 
























FFP 


10 


5 


9 


9 


4 


0 


2.44 


0.96 


0.91 


2 


0.01 


FPE 


17 


5 


6 


4 


5 


0 


2.45 


1.12 


1.25 


2 


0.68 


FPIS 


17 


5 


7 


6 


2 


0 


2.25 


0.94 


0.89 


2 


0.61 


FPR(P) 


16 


5 


7 


7 


2 


0 


2.29 


0.93 


0.87 


2 


0.49 


FPR(R) 


16 


5 


7 


6 


3 


0 


2.33 


0.99 


0.98 


2 


0.55 


FPIF 


13 


7 


7 


6 


4 


0 


2.29 


1.06 


1.12 


2 


0.44 


CPIF 


13 


5 


10 


5 


4 


0 


2.33 


0.99 


0.97 


2 


0.37 


CPAF 


14 


5 


10 


5 


3 


0 


2.26 


0.94 


0.89 


2 


0.43 


CPFF 


14 


5 


8 


6 


4 


0 


2.39 


1.01 


1.02 


2 


0.39 


T&M 


14 


4 


5 


6 


8 


0 


2.78 


1.10 


1.21 


3 


0.22 


Labor Hour 


14 


2 


6 


8 


7 


0 


2.87 


0.95 


0.90 


3 


0.07 


























Question 4 
























Pre-Award 


3 


9 


12 


12 


1 


0 


2.15 


0.84 


0.71 


2 


(0.28) 


Post-Award 


4 


5 


10 


12 


6 


0 


2.58 


0.94 


0.88 


3 


(0.45) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


4 


6 


12 


12 


3 


0 


2.36 


0.88 


0.78 


2 


(0.35) 


5b 


2 


6 


15 


13 


1 


0 


2.26 


0.77 


0.59 


2 


(0.51) 


5c 


2 


13 


12 


10 


0 


0 


1.91 


0.81 


0.65 


2 


(0.07) 


5d 


1 


16 


15 


4 


1 


0 


1.72 


0.77 


0.59 


2 


0.68 


5e 


7 


4 


12 


10 


4 


0 


2.47 


0.88 


0.78 


2 


(0.26) 


5f 


4 


6 


22 


5 


0 


0 


1.97 


0.58 


0.33 


2 


(0.74) 


Question 6 
























6a 


6 


9 


16 


5 


1 


0 


1.94 


0.76 


0.58 


2 


(0.01) 


6b 


4 


6 


8 


13 


6 


0 


2.58 


0.99 


0.97 


3 


(0.42) 


6c 


8 


6 


15 


7 


1 


0 


2.10 


0.76 


0.58 


2 


(0.14) 


6d 


6 


8 


14 


8 


1 


0 


2.06 


0.80 


0.64 


2 


(0.14) 


6e 


8 


7 


13 


7 


2 


0 


2.14 


0.86 


0.74 


2 


0.04 


6f 


5 


6 


19 


7 


0 


0 


2.03 


0.64 


0.41 


2 


(0.59) 


6g 


6 


3 


11 


13 


4 


0 


2.58 


0.83 


0.70 


3 


(0.51) 


Question 7 
























7a 


3 


11 


15 


6 


2 


0 


1.97 


0.86 


0.73 


2 


0.22 


7b 


3 


11 


20 


2 


1 


0 


1.79 


0.68 


0.46 


2 


0.12 


7c 


4 


12 


19 


1 


1 


0 


1.73 


0.66 


0.44 


2 


0.17 


Question 8 
























8a 


8 


5 


12 


10 


2 


0 


2.31 


0.83 


0.70 


2 


(0.20) 


8b 


9 


4 


14 


7 


3 


0 


2.32 


0.85 


0.72 


2 


(0.04) 


8c 


9 


6 


17 


4 


1 


0 


2.00 


0.71 


0.50 


2 


(0.04) 


8d 


10 


3 


19 


4 


1 


0 


2.11 


0.63 


0.40 


2 


(0.18) 


























Question 9 
























9 


10 


3 


14 


9 


1 


0 


2.30 


0.70 


0.49 


2 


(0.24) 


Question 10 
























10a 


2 


1 


13 


13 


8 


0 


2.80 


0.82 


0.67 


3 


(0.66) 


10b 


5 


1 


12 


13 


5 


1 


2.78 


0.86 


0.73 


3 


(0.48) 


10c 


3 


1 


10 


19 


4 


0 


2.76 


0.69 


0.47 


3 


(1.12) 


lOd 


4 


3 


17 


12 


1 


0 


2.33 


0.68 


0.46 


2 


(0.73) 
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Negotiation Preparation Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


33 


4 


23 


35 


5 


1 


2.65 


0.76 


0.58 


3 


(0.14) 


Validating 


23 


3 


29 


41 


5 


0 


2.62 


0.66 


0.44 


3 


(0.60) 


Program Est 


24 


13 


37 


25 


2 


0 


2.21 


0.74 


0.55 


2 


(0.24) 


Budgeting 


27 


14 


37 


20 


3 


0 


2.16 


0.77 


0.60 


2 


(0.05) 


ICE 


44 


6 


21 


24 


6 


0 


2.53 


0.82 


0.67 


3 


0.27 


Software 


38 


10 


22 


20 


11 


0 


2.51 


0.96 


0.92 


2 


0.25 


Repair Parts 


34 


3 


17 


37 


10 


0 


2.81 


0.74 


0.54 


3 


(0.22) 


R&D 


26 


9 


22 


24 


18 


2 


2.76 


1.03 


1.06 


3 


(0.09) 


Service 


28 


4 


13 


32 


23 


1 


3.05 


0.87 


0.76 


3 


(0.37) 


Hardware 


20 


6 


27 


39 


9 


0 


2.63 


0.78 


0.60 


3 


(0.54) 


Construction 


45 


3 


16 


24 


13 


0 


2.84 


0.84 


0.71 


3 


0.22 


Commodity 


50 


1 


5 


23 


21 


1 


3.31 


0.75 


0.57 


3 


0.23 


















































Question 2 
























CE 


23 


15 


25 


22 


14 


2 


2.53 


1.07 


1.15 


2 


0.07 


PD&RR 


21 


13 


24 


34 


8 


1 


2.50 


0.92 


0.85 


3 


(0.21) 


E&MD 


18 


9 


34 


35 


5 


0 


2.43 


0.76 


0.58 


2 


(0.50) 


PFD&OS 


20 


9 


26 


35 


11 


0 


2.59 


0.85 


0.72 


3 


(0.41) 


















































Question 3 
























FFP 


18 


7 


27 


37 


12 


0 


2.65 


0.83 


0.69 


3 


(0.52) 


FPE 


35 


3 


17 


33 


13 


0 


2.85 


0.78 


0.61 


3 


(0.16) 


FPIS 


37 


2 


20 


36 


6 


0 


2.72 


0.67 


0.45 


3 


(0.15) 


FPR(P) 


32 


3 


23 


36 


7 


0 


2.68 


0.71 


0.51 


3 


(0.27) 


FPR(R) 


34 


3 


21 


33 


10 


0 


2.75 


0.76 


0.58 


3 


(0.17) 


FPIF 


31 


2 


25 


34 


9 


0 


2.71 


0.72 


0.52 


3 


(0.28) 


CPIF 


28 


6 


22 


33 


12 


0 


2.70 


0.84 


0.70 


3 


(0.26) 


CPAF 


28 


7 


22 


33 


11 


0 


2.66 


0.85 


0.72 


3 


(0.24) 


CPFF 


25 


8 


22 


34 


12 


0 


2.66 


0.87 


0.75 


3 


(0.30) 


T&M 


29 


4 


14 


30 


23 


1 


3.04 


0.89 


0.79 


3 


(0.32) 


Labor Hour 


32 


3 


17 


24 


24 


1 


3.04 


0.91 


0.82 


3 


(0.1 9^ 
























1 


jouestion 4 






















i 


|Pre-Award 


13 


9 


32 


38 


9 


0 


2.53 


0.81 


0.66 


3 


(0.58)1 


jPost-Award 


19 


4 


20 


27 


28 


3 


3.07 


0.96 


0.91 


3 


(0.51^ 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


16 


8 


36 


30 


11 


0 


2.52 


0.83 


0.70 


2 


(0.41) 


5b 


14 


9 


41 


31 


6 


0 


2.39 


0.76 


0.58 


2 


(0.49) 


5c 


12 


20 


45 


19 


5 


0 


2.10 


0.81 


0.65 


2 


(0.10) 


5d 


15 


12 


56 


14 


4 


0 


2.12 


0.69 


0.47 


2 


(0.28) 


5e 


22 


2 


15 


41 


21 


0 


3.03 


0.75 


0.56 


3 


(0.70) 


5f 


15 


7 


50 


23 


6 


0 


2.33 


0.72 


0.52 


2 


(0.41) 


Question 6 
























6a 


22 


13 


46 


16 


4 


0 


2.14 


0.74 


0.55 


2 


(0.13) 


6b 


18 


7 


25 


36 


15 


0 


2.71 


0.86 


0.74 


3 


(0.52) 


6c 


14 


12 


47 


20 


8 


0 


2.28 


0.81 


0.66 


2 


(0.20) 


6d 


18 


8 


41 


26 


8 


0 


2.41 


0.79 


0.63 


2 


(0.34) 


6e 


19 


11 


35 


29 


7 


0 


2.39 


0.82 


0.68 


2 


(0.31) 


6f 


18 


17 


49 


12 


5 


0 


2.06 


0.77 


0.59 


2 


(0.03) 


6g 


19 


8 


36 


29 


9 


0 


2.48 


0.81 


0.66 


2 


(0.36) 


Question 7 
























7a 


12 


30 


50 


7 


2 


0 


1.79 


0.68 


0.46 


2 


0.00 


7b 


12 


30 


53 


5 


1 


0 


1.74 


0.61 


0.37 


2 


(0.23> 


7b 


14 


31 


52 


3 


1 


0 


1.70 


0.59 


0.35 


2 


(0.23)1 


Question 8 
























8a 


16 


15 


49 


14 


8 


0 


2.17 


0.82 


0.66 


2 


(0.03> 


8b 


16 


13 


52 


15 


5 


0 


2.14 


0.74 


0.54 


2 


(0.19) 


8c 


16 


25 


49 


7 


5 


0 


1.91 


0.77 


0.60 


2 


0.19 i 


8e 


17 


15 


52 


11 


6 


0 


2.10 


0.77 


0.59 


2 


(0.04) 


























Question 9 
























9 


30 


6 


37 


20 


8 


0 


2.42 


0.80 


0.63 


2 


(0.04) 


Question 10 
























10a 


15 


3 


38 


38 


7 


0 


2.57 


0.69 


0.48 


3 


(0.73) 


10b 


21 


0 


24 


43 


13 


0 


2.86 


0.67 


0.44 


3 


(0.73) 


10c 


18 


4 


26 


42 


11 


0 


2.72 


0.75 


0.56 


3 


(0.68) 


lOd 


19 


13 


35 


28 


6 


0 


2.33 


0.83 


0.68 


2 


(0-27) 
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Review or Audit Estimates Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


33 


6 


19 


27 


15 


1 


2.79 


0.93 


0.87 


3 


(0.04) 


Validating 


18 


9 


25 


36 


10 


3 


2.67 


0.95 


0.89 


3 


(0.27) 


Program Est 


28 


10 


28 


31 


4 


0 


2.40 


0.79 


0.62 


2 


(0.19) 


Budgeting 


29 


9 


33 


26 


4 


0 


2.35 


0.77 


0.59 


2 


(0.14) 


ICE 


52 


8 


15 


18 


8 


0 


2.53 


0.95 


0.90 


3 


0.63 


Software 


41 


11 


14 


23 


10 


2 


2.63 


1.06 


1.13 


3 


0.38 


Repair Parts 


34 


2 


16 


32 


17 


0 


2.96 


0.78 


0.61 


3 


(0.22) 


R&D 


28 


4 


16 


25 


25 


3 


3.10 


0.97 


0.94 


3 


(0.26) 


Service 


31 


2 


12 


26 


30 


0 


3.20 


0.82 


0.67 


3 


(0.37) 


Hardware 


25 


7 


29 


31 


9 


0 


2.55 


0.82 


0.67 


3 


(0.28) 


Construction 


39 


2 


17 


30 


13 


0 


2.87 


0.77 


0.60 


3 


(0.03) 


Commodity 


52 


0 


5 


24 


19 


1 


3.33 


0.68 


0.46 


3 


0.28 


















































Question 2 
























CE 


27 


17 


21 


19 


17 


0 


2.49 


1.07 


1.15 


2 


0.11 


PD&RR 


22 


13 


25 


32 


8 


1 


2.48 


0.93 


0.86 


3 


(0.16) 


E&MD 


0 


6 


27 


38 


5 


0 


2.55 


0.72 


0.52 


3 


(0.46) 


PFD&OS 


28 


5 


25 


33 


10 


0 


2.66 


0.80 


0.63 


3 


(0.27) 


















































Question 3 
























FFP 


30 


6 


30 


24 


11 


0 


2.56 


0.85 


0.72 


2 


(0.08) 


FPE 


42 


3 


24 


21 


11 


0 


2.68 


0.83 


0.69 


3 


0.20 


FPIS 


43 


3 


22 


25 


8 


0 


2.66 


0.78 


0.61 


3 


0.17 


FPR(P) 


42 


3 


26 


23 


7 


0 


2.58 


0.76 


0.58 


3 


0.17 


FPR(R) 


43 


3 


23 


23 


9 


0 


2.66 


0.80 


0.64 


3 


0.20 


FPIF 


35 


5 


27 


25 


9 


0 


2.58 


0.82 


0.67 


3 


0.01 


CPIF 


33 


4 


20 


32 


12 


0 


2.76 


0.81 


0.65 


3 


(0.17) 


CPAF 


36 


5 


17 


32 


11 


0 


2.75 


0.82 


0.68 


3 


(0.07) 


CPFF 


35 


5 


15 


35 


11 


0 


2.79 


0.81 


0.65 


3 


(0.13)1 


T&M 


36 


3 


9 


26 


25 


2 


3.22 


0.89 


0.78 


3 


(0.15) 


Labor Hour 


38 


2 


10 


24 


25 


2 


3.24 


0.87 


0.75 


3 


(0.09) 
























1 


buestion 4 
























Pre-Award 


19 


12 


35 


31 


4 


0 


2.33 


0.78 


0.61 


2 


(0.38) 


Post-Award 


23 


3 


15 


25 


28 


7 


3.27 


1.00 


0.99 


3 


(0.43) 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


14 


11 


32 


30 


14 


0 


2.54 


0.91 


0.82 


3 


(0.37) 


5b 


12 


9 


51 


24 


5 


0 


2.28 


0.72 


0.52 


2 


(0.44) 


5c 


14 


18 


47 


19 


3 


0 


2.08 


0.75 


0.56 


2 


(0.24) 


5d 


14 


17 


39 


26 


5 


0 


2.22 


0.82 


0.68 


2 


(0.25) 


5e 


24 


1 


17 


38 


21 


0 


3.03 


0.74 


0.54 


3 


(0.62) 


5f 


15 


10 


48 


21 


7 


0 


2.29 


0.78 


0.60 


2 


(0.28) 


Question 6 
























6a 


21 


12 


39 


24 


5 


0 


2.28 


0.79 


0.62 


2 


(0.22) 


6b 


16 


4 


28 


37 


16 


0 


2.76 


0.81 


0.65 


3 


(0.64) 


6c 


19 


12 


40 


19 


11 


0 


2.35 


0.89 


0.79 


2 


(0.09) 


6d 


19 


14 


41 


23 


4 


0 


2.21 


0.78 


0.60 


2 


(0.24) 


6e 


20 


16 


31 


29 


5 


0 


2.28 


0.85 


0.72 


2 


(0.21) 


6f 


22 


19 


41 


16 


3 


0 


2.04 


0.77 


0.59 


2 


(0.03) 


6g 


23 


7 


29 


33 


9 


0 


2.56 


0.81 


0.66 


3 


(0.36) 


Question 7 
























7a 


14 


36 


42 


6 


3 


0 


1.72 


0.74 


0.54 


2 


0.33 


7b 


14 


32 


50 


4 


1 


0 


1.70 


0.61 


0.37 


2 


(0.16) 


7c 


16 


36 


45 


3 


1 


0 


1.64 


0.61 


0.37 


2 


(0.03) 


Question 8 
























8a 


20 


17 


35 


18 


11 


0 


2.28 


0.95 


0.89 


2 


0.04 


8b 


21 


20 


31 


19 


10 


0 


2.24 


0.96 


0.93 


2 


0.10 


8c 


17 


26 


43 


10 


5 


0 


1.93 


0.81 


0.66 


2 


0.21 


8® 


IT 


17 


49 


13 


4 


0 


2.05 


0.74 


0.55 


2 


(0.09) 


























Question 9 
























9 


33 


9 


36 


18 


4 


1 


2.29 


0.82 


0.68 


2 


0.18 


Question 10 
























10a 


17 


5 


40 


30 


9 


0 


2.51 


0.76 


0.58 


2 


(0.48) 


10b 


26 


1 


18 


46 


9 


1 


2.88 


0.67 


0.45 


3 


(0.55) 


10c 


19 


4 


28 


40 


9 


1 


2.70 


0.78 


0.60 


3 


(0.54) 


lOd 


20 


8 


37 


29 


6 


1 


2.44 


0.82 


0.67 


2 


(0.26) 
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Set Corporate Policy Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variance 


Median 


Skew 


Question 1 
























Benchmarking 


0 


3 


2 


0 


0 


0 


1.40 


0.49 


0.24 


1 


0.61 


Validating 


0 


2 


3 


0 


0 


0 


1.60 


0.49 


0.24 


2 


._.(Q61) 


Program Est 


0 


3 


1 


1 


0 


0 


1.60 


0.80 


0.64 


1 


1.26 


Budgeting 


0 


2 


3 


0 


0 


0 


1.60 


0.49 


0.24 


2 


(0.61) 


ICE 


1 


2 


0 


2 


0 


0 


2.00 


1.00 


1.00 


2 


0.17 


Software 


0 


3 


1 


1 


0 


0 


1.60 


0.80 


0.64 


1 


1.26 


Repair Parts 


1 


2 


1 


0 


2 


0 


2.40 


1.30 


1.69 


3 


(0.05) 


R&D 


1 


1 


1 


1 


1 


0 


2.50 


1.12 


1.25 


3 


0.00 


Service 


1 


0 


0 


3 


1 


0 


3.25 


0.43 


0.19 


3 


(1.75) 


Hardware 


1 


2 


0 


1 


1 


0 


2.25 


1.30 


1.69 


2 


0.52 


Construction 


0 


1 


0 


2 


2 


0 


3.00 


1.10 


1.20 


3 


(1.36) 


Commodity 


1 


0 


0 


2 


2 


0 


3.50 


0.50 


0.25 


4 


(1.74) 


















































Question 2 
























CE 


0 


4 


1 


0 


0 


0 


1.20 


0.40 


0.16 


1 


2.24 


PD&RR 


0 


3 


2 


0 


0 


0 


1.40 


0.49 


0.24 


1 


0.61 


E&MD 


0 


2 


2 


1 


0 


0 


1.80 


0.75 


0.56 


2 


0.51 


PFD&OS 


0 


2 


2 


1 


0 


0 


1.80 


0.80 


0.64 


2 


0.05 


















































Question 3 
























FFP 


1 


1 


1 


2 


0 


0 


2.25 


0.83 


0.69 


3 


(0.54) 


FPE 


2 


1 


2 


1 


0 


0 


2.00 


0.82 


0.67 


2 


0.54 


FPIS 


2 


1 


1 


1 


0 


0 


2.00 


0.82 


0.67 


2 


0.54 


FPR(P) 


2 


1 


1 


1 


0 


0 


2.00 


0.82 


0.67 


2 


0.54 


FPR(R) 


2 


1 


1 


1 


0 


0 


2.00 


0.82 


0.67 


2 


0.54 


FPIF 


2 


1 


1 


1 


0 


0 


2.00 


0.82 


0.67 


2 


0.54 


CPIF 


2 


2 


0 


1 


0 


0 


1.67 


0.94 


0.89 


1 


1.36 


CPAF 


2 


2 


0 


1 


0 


0 


1.67 


0.94 


0.89 


1 


1.36 


bpFF 


2 


2 


0 


1 


0 


0 


1.67 


0.94 


0.89 


1 


1.36 


t&M 


2 


2 


0 


1 


0 


0 


1.67 


0.94 


o.eo 


1 


1.36 


Labor Hour 


3 


0 


0 


2 


0 


0 


3.00 


0.00 


0.00 


3 


0.61 


























buestion 4 
























Pre-Award 


0 


2 


3 


0 


0 


0 


1.60 


0.49 


0.24 


2 


(0.61) 


|Post-Award 


0 


1 


3 


1 


0 


0 


2.00 


0.63 


0.40 


2 


0.00 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variance 


Median 


Skew 


5a 


0 


3 


2 


0 


0 


0 


1.40 


0.49 


0.24 


1 


0.61 


5b 


0 


3 


2 


0 


0 


0 


1.40 


0.49 


0.24 


1 


0.61 


5c 


0 


4 


1 


0 


0 


0 


1.20 


0.40 


0.16 


1 


2.24 


5d 


0 


4 


1 


0 


0 


0 


1.20 


0.40 


0.16 


1 


2.24 


5e 


2 


0 


2 


1 


0 


0 


2.33 


0.47 


0.22 


2 


(0.17) 


5f 


1 


3 


1 


0 


0 


0 


1.25 


0.43 


0.19 


1 


0.00 


Question 6 
























6a 


1 


3 


2 


0 


0 


0 


1.40 


0.43 


0.19 


1 


0.00 


6b 


0 


2 


1 


1 


1 


0 


2.20 


1.17 


1.36 


2 


0.54 


6c 


0 


2 


2 


1 


0 


0 


1.80 


0.75 


0.56 


2 


0.51 


6d 


0 


3 


1 


1 


0 


0 


1.60 


0.80 


0.64 


1 


1.26 


6e 


0 


3 


1 


1 


0 


0 


1.60 


0.80 


0.64 


1 


1.26 


6f 


0 


2 


2 


1 


0 


0 


1.80 


0.75 


0.56 


2 


0.51 


6g 


1 


1 


2 


1 


0 


0 


2.00 


0.83 


0.69 


3 


(0.54) 


Question 7 
























7a 


0 


2 


1 


2 


0 


0 


2.00 


0.89 


0.80 


2 


0.00 


7e 


i: 


3 


2 


0 


0 


0 


1.40 


0.4S 


0.24 


1 


0.61 i 


7c 


1 


3 


1 


0 


0 


0 


1.25 


0.43 


0.19 


1 


0.001 


'Question 8 
























(8a 


0 


1 


3 


0 


1 


0 


2.20 


0.98 


0.96 


2 


^29 


;8b 


1 


1 


2 


0 


1 


0 


2.25 


1.09 


1.19 


2 


0.55 


Sb 


1 


2 


3 


0 


0 


0 


1.60 


0.43 


0.19 


2 


(1.26^ 


l8d 


1 


0 


3 


1 


0 


0 


2.25 


0.43 


0.19 


2 


(1.29^ 
























Question 9 
























9 


1 


0 


3 


1 


0 


0 


2.25 


0.43 


0.19 


2 


(1.29) 


Question 10 
























10a 


0 


0 


4 


1 


0 


0 


2.20 


0.40 


0.16 


2 


2.24 


10b 


0 


0 


3 


2 


0 


0 


2.40 


0.49 


0.24 


2 


0.61 


10c 


0 


1 


1 


3 


0 


0 


2.40 


0.80 


0.64 


3 


(1.26) 


lOd 


1 


1 


3 


0 


0 


0 


1.75 


0.43 


0.19 


2 


(1-26) 
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Federal Government Policy Group Data 





No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


Question 1 
























Benchmarking 


8 


2 


10 


6 


2 


0 


2.40 


0.80 


0.64 


2 


(0.08) 


Validating 


6 


3 


8 


7 


4 


0 


2.55 


0.94 


0.88 


3 


(0.19) 


Program Est 


4 


7 


9 


7 


1 


0 


2.08 


0.86 


0.74 


2 


(0.08) 


Budgeting 


5 


5 


9 


8 


1 


0 


2.22 


0.83 


0.69 


2 


(0.24) 


ICE 


11 


6 


6 


3 


2 


0 


2.06 


1.00 


1.00 


2 


0.71 


Software 


6 


4 


6 


7 


4 


1 


2.64 


1.11 


1.23 


3 


0.01 


Repair Parts 


7 


0 


4 


12 


5 


0 


3.05 


0.65 


0.43 


3 


(0.69) 


R&D 


7 


1 


5 


8 


5 


2 


3.10 


1.02 


1.04 


3 


(0.23) 


Service 


5 


1 


6 


9 


7 


0 


2.96 


0.86 


0.74 


3 


(0.67) 


Hardware 


3 


3 


7 


12 


3 


0 


2.60 


0.85 


0.72 


3 


(0.69) 


Construction 


5 


0 


8 


11 


4 


0 


2.83 


0.70 


0.49 


3 


(0.77) 


Commodity 


11 


0 


2 


7 


8 


0 


3.35 


0.68 


0.46 


3 


(0.19) 


















































Question 2 
























CE 


4 


6 


6 


8 


4 


0 


2.42 


1.04 


1.08 


3 


(0.14) 


PD&RR 


4 


3 


11 


7 


3 


0 


2.42 


0.86 


0.74 


2 


(0.29) 


E&MD 


4 


2 


12 


8 


2 


0 


2.42 


0.76 


0.58 


2 


(0.49) 


PFD&OS 


3 


2 


11 


7 


5 


0 


2.60 


0.89 


0.80 


2 


(0.40) 


















































Question 3 
























FFP 


5 


2 


9 


9 


3 


0 


2.57 


0.82 


0.68 


3 


(0.46) 


FPE 


7 


0 


7 


11 


3 


0 


2.81 


0.66 


0.44 


3 


(0.58) 


FPIS 


6 


1 


8 


10 


3 


0 


2.68 


0.76 


0.58 


3 


(0.53) 


FPR(P) 


6 


1 


7 


11 


3 


0 


2.73 


0.75 


0.56 


3 


(0.59) 


FPR(R) 


6 


1 


8 


10 


3 


0 


2.68 


0.76 


0.58 


3 


(0.53) 


FPIF 


6 


0 


10 


9 


3 


0 


2.68 


0.70 


0.49 


3 


(0.55) 


CPIF 


6 


2 


6 


10 


4 


0 


2.73 


0.86 


0.74 


3 


(0.46) 


CPAF 


7 


2 


7 


8 


3 


1 


2.71 


0.98 


0.97 


3 


(0.07) 


CPFF 


7 


2 


7 


8 


3 


1 


2.71 


0.98 


0.97 


3 


(0.07)1 


T&M 


6 


0 


2 


13 


6 


1 


3.27 


0.69 


0.47 


3 


(0.82) 


Labor Hour 


6 


0 


3 


10 


8 


1 


3.32 


0.76 


0.58 


3 


(0.76) 


























Question 4 
























Pre-Award 


4 


6 


9 


9 


0 


0 


2.13 


0.78 


0.61 


2 


(0.43)1 


Post-Award 


5 


1 


6 


11 


5 


0 


2.87 


0.80 


0.64 


3 


(0.72)1 
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Question 5 


No Response 


#1 


#2 


#3 


#4 


#5 


Mean 


Std Dev 


Variation 


Median 


Skew 


5a 


2 


3 


10 


9 


4 


0 


2.54 


0.89 


0.79 


3 


(0.42) 


5b 


3 


1 


13 


8 


3 


0 


2.52 


0.75 


0.57 


2 


(0.54) 


5c 


2 


7 


14 


3 


2 


0 


2.00 


0.83 


0.69 


2 


0.30 


5d 


3 


5 


13 


5 


2 


0 


2.16 


0.83 


0.69 


2 


(0.05) 


5e 


5 


1 


7 


10 


5 


0 


2.83 


0.82 


0.67 


3 


(0.64) 


5f 


5 


3 


14 


4 


2 


0 


2.22 


0.78 


0.60 


2 


(0.13) 


Question 6 
























6a 


4 


4 


14 


5 


1 


0 


2.13 


0.73 


0.53 


2 


(0.29) 


6b 


5 


1 


9 


9 


4 


0 


2.70 


0.80 


0.65 


3 


(0.54) 


6c 


2 


3 


12 


9 


2 


0 


2.38 


0.79 


0.62 


2 


(0.46) 


6d 


3 


2 


11 


8 


4 


0 


2.56 


0.85 


0.73 


2 


(0.45) 


6e 


2 


3 


11 


8 


4 


0 


2.50 


0.89 


0.79 


2 


(0.33) 


6f 


3 


7 


15 


2 


1 


0 


1.88 


0.71 


0.51 


2 


0.07 


6g 


4 


1 


11 


9 


3 


0 


2.58 


0.76 


0.58 


3 


(0.60) 


Question 7 
























7a 


3 


9 


12 


3 


1 


0 


1.84 


0.78 


0.61 


2 


0.25 


7b 


2 


9 


14 


2 


1 


0 


1.81 


0.73 


0.54 


2 


0.33 


7c 


3 


10 


13 


1 


1 


0 


1.72 


0.72 


0.52 


2 


0.41 


Question 8 
























8a 


3 


2 


19 


2 


2 


0 


2.16 


0.67 


0.45 


2 


(0.14) 


isb 


2 


1 


20 


2 


3 


0 


2.27 


0.71 


0.50 


2 


0.09 


be 


2 


5 


18 


2 


1 


0 


1.96 


0.65 


0.42 


2 


(0.08) 


8d 


3 


2 


17 


5 


1 


0 


2.20 


0.63 


0.40 


2 


(0.54) 


























Question 9 
























9 


8 


2 


11 


4 


2 


1 


2.45 


0.97 


0.95 


2 


0.31 


Question 10 
























10a 


4 


1 


13 


8 


2 


0 


2.46 


0.71 


0.50 


2 


(0.59) 


10b 


4 


0 


7 


11 


5 


1 


3.00 


0.82 


0.67 


3 


(0.69) 


10c 


4 


0 


9 


6 


8 


1 


3.04 


0.93 


0.87 


3 


(0.54) 


lOd 


6 


4 


6 


9 


2 


1 


2.55 


1.03 


1.07 


3 


0.00 
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